Anthropic's Amodei outlines plan to pace AI development
In a new blog post, Anthropic CEO Dario Amodei echoed Sam Altman's earlier suggestion that it may be time to "pace" AI development, and went further by laying out three broad strategies for doing so. He said two things convinced him a more cautious approach was needed: the OpenAI-HuggingFace hack, and AI's advancing "drastically faster" in recent months, particularly its growing ability to build the next generation of AI. "We must slow the pace at which we improve the capabilities of AI models," he wrote, adding that "progress will still seem fast, and we must make wise use of the time we gain."
The post landed amid an intensifying safety debate. Anthropic researcher Jacob Coxon resigned this week, writing that leading AI companies are "gambling with our lives" while the people building the technology "earnestly believe it could kill us all by the end of the decade," a claim he said others at Anthropic share. Amodei's post did not explicitly mention Coxon's resignation or concerns.
Amodei's first step is embedding "evaluators" from third-party organizations such as METR inside frontier AI companies, to verify that pacing and safety commitments are actually being followed and that safety incidents get reported; he noted OpenAI was recently criticized for not reporting an incident in which its AI agents took over a German wiki forum. He compared these evaluators to bank regulators, and said Anthropic is "unilaterally committing" to giving them company badges, desks and laptops, with access "mostly comparable to what internal risk assessment teams have," aside from exceptions required by law or contract, and he called on governments to require other frontier companies to match. Altman called it a "good idea" and said OpenAI would do the same, adding "we'll have more to share soon." Elon Musk posted, "Dario is right."
The second strategy calls for leading AI companies "within democratic countries" to coordinate on common safety standards and limits on the rate of unchecked progress. Amodei acknowledged that companies are reportedly wary of antitrust scrutiny over such coordination, and said the US government should mediate or at least enable the talks and issue "a narrow waiver for certain kinds of safety conversations," without necessarily participating itself.
On the argument that slowing down cedes ground to China, Amodei said steps like blocking sales of powerful chips and semiconductor manufacturing equipment to Chinese firms, plus a crackdown on model distillation, could "slow China's progress enough to widen America's lead significantly over the next 3-5 years." His third strategy, global coordination, calls for the US and its allies to attempt cooperation even with authoritarian governments including China, despite "stark limits on what can be achieved," aiming at narrow bans such as "prohibiting certain narrow and obviously dangerous uses of AI, such as using AI for the production of biological weapons or allowing users to do so."
Amodei has faced criticism as a "doomer" whose warnings feed an AI backlash; he responded that the backlash is "fundamentally a crisis of trust" stemming from public skepticism of tech companies and government. Journalist Brian Merchant pushed back harder, writing he has yet to see "a credible, step-by-step documentation" of how AI could move from self-recursive improvement to wiping out humanity, and calling Amodei's proposals a likely case of "regulatory capture" that would mainly serve Anthropic and OpenAI. Amodei maintains he still believes AI can "enormously improve the quality of human life," but only "if we build the technology in the right way."
Key facts
- Amodei outlined three strategies to pace AI development and committed Anthropic unilaterally to the first: embedding third-party evaluators like METR inside the company with badges, desks and laptops.
- Sam Altman called it a "good idea" and said OpenAI would do the same; Elon Musk posted "Dario is right."
- The post came after Anthropic researcher Jacob Coxon resigned, saying AI companies are "gambling with our lives" while insiders believe it could kill everyone by the end of the decade.
- Amodei's second strategy calls for coordinated safety standards among democratic-country AI firms, and his third is global coordination that includes cooperation with China; separately, he argued that limiting Chinese firms' access to chips and manufacturing equipment could widen the US lead by 3 to 5 years.
- Journalist Brian Merchant called the proposal "regulatory capture," arguing it would mainly serve Anthropic and OpenAI rather than address real AI harms.
Why it matters
Anthropic CEO Dario Amodei published a blog post calling on the AI industry to "pace the frontier," the clearest public commitment yet from a leading AI lab to voluntarily slow its own capability gains. He named two triggers: the OpenAI-HuggingFace hack and what he called AI's "advancing drastically faster" pace in recent months, especially its growing ability to build the next generation of AI models. Rather than only urging others to slow down, Amodei said Anthropic is unilaterally committing to the first of three concrete steps, and OpenAI CEO Sam Altman immediately said OpenAI would follow, with SpaceX's Elon Musk endorsing the move too. That makes this a rare moment of public alignment between rival AI labs on a safety commitment, not just a rhetorical gesture.
Who it affects
The proposal falls first on Anthropic and OpenAI, whose leaders have now stated intent to embed independent evaluators inside their operations. It also affects third-party evaluation groups such as METR, which Amodei named as a model for the outside overseers he wants inside frontier labs. Governments are drawn in too: Amodei is asking them to require other frontier companies to match Anthropic's commitment, to mediate coordination talks among AI companies, and to issue narrow antitrust waivers so competitors can discuss safety without legal risk. The debate was sharpened by Anthropic researcher Jacob Coxon, who resigned saying the leading AI companies are "gambling with our lives" while insiders privately believe the technology "could kill us all by the end of the decade."
How to use it
For now there is nothing to adopt, only signals to watch. The concrete test is whether Anthropic follows through on giving outside evaluators like METR real company badges, desks and laptops with access "comparable to what internal risk assessment teams have," and whether Altman's promised follow-up ("we'll have more to share soon") turns into an actual OpenAI program rather than a supportive quote. On the geopolitical track, Amodei's suggested next steps are export controls on advanced chips and manufacturing equipment to Chinese firms and a crackdown on model distillation, steps he says could widen the US lead over China over the next 3 to 5 years; whether Washington acts on either is the thing to track.
How solid is it
The account rests on Amodei's own blog post and on real-time reactions from Altman and Musk, quoted directly, giving the core claims solid, on-the-record footing. What is less solid is the follow-through: Altman's response is a statement of intent, not a rolled-out program, and no other frontier AI company has yet signed on to any of the three proposals. The source does not give Coxon's resignation date or the blog post's exact publication date, and it does not detail the scope of the OpenAI-HuggingFace hack or the German wiki forum incident it references as context, so those remain background pointers rather than verified specifics.
Risks and caveats
Amodei's framing has drawn direct pushback. Journalist Brian Merchant said he has yet to see "a credible, step-by-step documentation" of how AI could plausibly move from self-improvement to human extinction, and argued that proposals like Amodei's "would likely only wind up serving Anthropic and OpenAI," calling it regulatory capture in action. Amodei's own third proposal, global coordination that includes "cooperation with China," is one he admits faces "stark limits on what can be achieved," and even Anthropic's unilateral evaluator commitment carries built-in exceptions "when required by law or contracts," leaving the scope of outside access undefined until it is tested in practice.
“We must slow the pace at which we improve the capabilities of AI models.”
— Dario Amodei, Anthropic CEO