AI labs need coordinated plan to halt development if risks rise, Anthropic says

Frontier AI developers should establish a coordinated and verifiable method to slow or temporarily pause development if advanced systems begin improving themselves faster than society can manage the associated risks, according to AI startup Anthropic.

The company highlighted the potential risks of "full recursive self-improvement," where AI systems could build their own successors, stating it might increase the chances of humans losing control. Anthropic noted that as of May, over 80 percent of the code merged into its own system was authored by Claude, illustrating the rapid pace of AI advancement. "It would be good for the world to have the option to slow or temporarily pause frontier AI development to enable societal structures and alignment research to keep up with the advance of the technology," Anthropic stated.

“

If systems are capable of fully building their own successors, the ways we secure them, monitor them, and shape their behavior all grow much more important.

However, Anthropic cautioned that unilateral or poorly coordinated slowdowns could be counterproductive if less cautious actors continue to advance, potentially compromising overall safety. A meaningful pause, the company suggested, would require agreement among multiple well-resourced labs operating at the technological frontier. It would also necessitate clear rules on the conditions that would trigger or lift such a pause and who would oversee the process.

Anthropic's research arm plans to study and help build the systems necessary to support a slowdown. The company intends to convene discussions in the coming months involving policymakers, researchers, civil society groups, and other AI firms. These discussions will aim to examine key questions, including how to manage AI-related risks such as recursive self-improvement and how to improve coordination mechanisms for global AI development.

“

It would be good for the world to have the option to slow or temporarily pause frontier AI development to enable societal structures and alignment research to keep up with the advance of the technology.