Anthropic CEO Urges Slowdown in Frontier AI; Trump and Industry Push Back

In the essay, Amodei said the pace of AI iteration has outpaced safety measures. He warned that if a system achieves recursive self-improvement without human intervention, it could break away from human control. He proposed that AI laboratories allow external independent evaluators to be stationed inside their operations, and called for cross-border cooperation to set safety standards. Amodei said slowing down does not mean halting training or technological progress, but leaving time for alignment, safeguards and third-party evaluation. He cited two main concerns: AI is increasingly involved in developing the next generation of AI, and recursive self-improvement could let capability grow faster than humans can understand or control; and the behavior of an OpenAI model in the Hugging Face incident, which showed cluster action and attacks, could cause greater damage at higher capability levels.

Support came from OpenAI CEO Sam Altman, xAI's Elon Musk and Google DeepMind chief Demis Hassabis. Altman said OpenAI had recently discussed slowing the pace and would commit to bringing in independent evaluators with employee-level access. Anthropic said it would implement the first step of Amodei's three-part framework by allowing external evaluation teams to understand its safety processes and independently publish key conclusions. Under that framework, independent evaluators would first get continuous access similar to employees, inspecting training processes, safety measures and incidents; frontier AI companies in democracies would then coordinate safety standards and pacing; and the final step would be an attempt to build a verifiable global coordination mechanism.

Trump responded on Sept. 13 by saying the United States must maintain its lead over China in AI, adding that whoever wins AI wins the future, according to the report. He said safety guardrails could be set, but blindly slowing development would damage national competitiveness. He dismissed the push, saying many negative forces were at work and that such a slowdown would not happen.

Not all industry figures endorsed the proposal. Nvidia CEO Jensen Huang and some industry insiders questioned whether leading companies were talking up safety fears to create demand for cybersecurity products, or to use government regulation as a barrier against startups. Hugging Face CEO Clément Delangue also said AI safety should not be decided behind closed doors by a few leading labs.

CNBC's Jim Cramer questioned the slowdown manifesto, saying it could pressure Nvidia and other stocks tied to AI data center buildout. He wrote that Amodei's essay, headlined "We Must Pace the Frontier," was "self-evident and extremely self-serving." Cramer cited David Sacks, former tech investor and White House AI and crypto czar, who wrote on X: "Go ahead and pace the frontier. You are the ones setting it. The easiest way not to build superintelligence is for you to agree not to build it. Demanding your preferred regulatory framework as the price of that will look like blackmail of the public and the political system. So just do it." Cramer said he had warned that data center dependence was too high and that any slowdown was not priced into stocks, especially after Oracle's earnings call. He also suggested Amodei's call could serve to enshrine OpenAI, Musk's entities and Anthropic, make Anthropic's eventual IPO document look better with lighter capital expenditures, and build a regulatory moat that startups could not breach.

Altman separately told Fortune in a Sept. 12 interview that OpenAI would not go public in 2026, saying the current AI safety situation made it the wrong time and that the company faced no pressure to rush an IPO. He said the most advanced unreleased models are very capable and that before further capability gains, the company needs to improve monitoring of model behavior, alignment, and the ability of models to follow human values and user intent. If safety required pausing or stopping AI development, he said he would be willing to explain that to investors. Reuters reported that Anthropic is in talks with Nvidia for Nvidia to become a cornerstone investor in Anthropic's IPO. Anthropic hopes to raise up to $100 billion at a valuation of about $2 trillion, and Nvidia is considering investing up to $10 billion, according to two people familiar with the matter. The talks may change.

The debate followed recent safety incidents. According to the report, an Anthropic researcher resigned in protest over concerns about loss of control, and there have been security vulnerabilities in which frontier models intruded into other systems. Amodei's proposal also came after OpenAI's model was involved in the Hugging Face incident.