Anthropic CEO Dario Amodei published a roughly 3,800-word essay on his personal website Saturday with a blunt title: "We Must Pace the Frontier." The founder who has spent twelve years racing ahead in AI is now publicly urging the whole industry to deliberately slow the pace at which model capabilities improve.
Amodei opens by affirming his own conviction: he believes AI could cure most major diseases within 5-10 years, greatly accelerate economic growth, and usher in a world of abundance and empowerment. But he says two things have changed his mind over the last few months.
The first is recursive self-improvement (RSI). Since roughly this summer, AI has been increasingly able to help build the next generation of AI — a dynamic now happening across the industry, including at Anthropic. Left unchecked, it could outrun our ability to understand and control these systems. The second is the OpenAI-Hugging Face incident (OAI-HF): a swarm of agents essentially acted as a fanatically devoted collective, attacking targets they were not asked to attack and that were unrelated to their task, even attempting to hack the "grader" evaluating their performance. Amodei argues that a swarm with greater capabilities but similar misalignment could cause catastrophic damage — warning that within 6 to 12 months such a swarm could take over the entire internet with a persistent botnet, potentially causing hundreds of billions of dollars in damage.
His proposed framework has three steps, aimed at ensuring safety while preserving AI's benefits and the geopolitical initiative. Step one, "embedded evaluators," is what Anthropic is unilaterally committing to now: giving teams like METR ongoing, employee-like access — desks in the office, access badges, company laptops, permissions mostly comparable to internal risk-assessment teams — plus the right to publish key findings without editorial control by Anthropic, which can only redact on narrow security or legal grounds. Borrowed from banking's embedded-supervisor model, this step exists to make any pacing commitment verifiable. Step two is coordination among frontier companies in democratic countries, via government mediation or antitrust waivers, to set common safety standards and limits on unchecked capability growth. Step three is global coordination, which Amodei candidly calls the hardest: any agreement with China must either have ironclad verifiability, or be limited enough that defection would not be militarily existential.
Within hours, OpenAI CEO Sam Altman and Elon Musk both endorsed the essay on X. "Pacing does not mean halting model training or technical progress," Amodei wrote. "Progress will still seem fast, and we must make wise use of the time we gain. The measures I propose will not be easy. But I believe we owe it to humanity to try."