Anthropic calls for AI caution, but the market demands more AI development

The US-based AI company Anthropic, creator of the Claude AI model, has called for a pause in artificial intelligence development, suggesting a "global freeze" to allow humanity time to adapt. This call comes amidst rapid advancements where AI, like Claude, is increasingly capable of writing its own code and proposing experiments, narrowing the human role in development.

“

if it were possible to slow the development of this technology … we think that would likely be a good thing

In a blog post, Anthropic stated that if a mechanism for global coordination existed, slowing AI development "would likely be a good thing." However, the company acknowledges the unlikelihood of such a coordinated global effort, especially one that would include nations like China and Russia. This statement contrasts with Anthropic's own recent development of Mythos, a powerful AI model designed for computer security tasks, including hacking.

“

the human role is narrowing at each step in the AI development process

Recognizing the potential risks, Anthropic has not released Mythos publicly. Instead, it has been confined to a select group of insiders within an initiative called Project Glasswing. This project aims to develop defenses against sophisticated AI threats by allowing a chosen few to understand and prepare for capabilities like those demonstrated by Mythos. Recently, Australian firms were invited to join this exclusive club, alongside 150 others, presumably from democratic nations.

“

strikingly capable at computer security tasks

Anthropic CEO Dario Amodei has framed Project Glasswing as an effort to ensure democracies maintain an edge over autocracies. The initiative draws parallels to historical efforts like the Bretton Woods conference, which established a post-war economic system, though Glasswing is entirely private sector-driven, unlike the government-led Bretton Woods. The article draws a historical line from the Colossus computer of 1944 to Mythos, suggesting a continuous evolution towards self-building machines and the impending singularity.

“

aimed at making sure democracies stay ahead of autocracies