Anthropic calls for global pause on advanced AI development

Artificial intelligence company Anthropic has called for a global pause on the development of the most powerful AI systems, warning that current models show signs of potentially escaping human control. The San Francisco-based firm suggested that a worldwide slowdown in cutting-edge AI development would be "likely a good thing." However, Anthropic cautioned that if only one company halted progress, rivals would simply accelerate their own development.

"We believe it would be good for the world to have the option to slow or temporarily pause frontier AI development to enable societal structures and alignment research to keep up with the advance of the technology," Anthropic stated in a report. The company emphasized that a successful pause would require simultaneous agreement from multiple major AI companies across different countries, particularly the US and China, under verifiable rules.

Anthropic acknowledged the difficulty of achieving such a global coordination mechanism, noting that companies and governments would face "difficult decisions about safety while under competitive and geopolitical pressures." The proposal has already encountered pushback from industry peers and US officials, who view Anthropic's focus on worst-case scenarios as a potential strategy to hinder competitors under the guise of safety concerns.

“

We believe it would be good for the world to have the option to slow or temporarily pause frontier AI development to enable societal structures and alignment research to keep up with the advance of the technology.

The White House has recognized the capabilities of Anthropic's Mythos model, which is not publicly available due to its cybersecurity features and is restricted to a select group of vetted organizations. However, the proposal faces an uphill battle in Washington and Silicon Valley, where a slowdown in AI development is seen as a risk that could cede a strategic advantage to China in what is widely considered the defining technological race of the century.

Anthropic plans to convene government officials, scientists, advocacy groups, and competing AI firms in the coming months to explore how such a coordination system could function. The call for collaboration comes as internal data indicates AI is significantly accelerating its own development, creating a feedback loop that could lead to "recursive self-improvement", an AI system capable of enhancing itself.

“

Without a global coordination mechanism, companies and governments will have to make difficult decisions about safety while under competitive and geopolitical pressures.