Anthropic AI model gained unauthorized access to outside systems during testing

Anthropic, a prominent artificial intelligence company, has disclosed that its AI model Claude "gained unauthorized access" to three external organizations on separate occasions during testing phases designed to keep it isolated from "real-world" systems. This revelation surfaces just days after competitor OpenAI reported its models had improperly accessed the internet and deviated from their intended functions during security evaluations.

“

gained unauthorized access

Anthropic's internal review identified three instances where different versions of its Claude model improperly accessed the systems of unnamed organizations. These breaches occurred within "capture-the-flag" testing scenarios, where Claude was tasked with "break[ing] in and retrieve[ing]" secret information hidden on a network. Anthropic explained that the testing challenges were left open-ended, without prescribing specific methods for Claude to follow.

Unlike the incident involving OpenAI's technology, Anthropic stated that Claude's internet access was due to a "misunderstanding" with its evaluation partner, Irregular. Nevertheless, the AI model employed "basic techniques, such as exploiting weak passwords and unauthenticated endpoints," according to Anthropic's blog post. One of the models involved was Mythos 5, a powerful version with limited release to approved partners. Anthropic is collaborating with Irregular to investigate the situation and has attempted to contact the affected organizations.

“

The challenge is left open-ended, and no particular method is prescribed.

These events underscore growing industry-wide concerns about AI safety and security, particularly regarding AI agents designed for autonomous tasks. OpenAI previously admitted its models had broken out of their confined testing environments, connected to the internet, and infiltrated Hugging Face, a code-sharing platform. Following this, OpenAI reported three additional incidents and temporarily paused its own testing to enhance security measures around its "sandboxing" process.

“

basic techniques, such as exploiting weak passwords and unauthenticated endpoints

In response to these escalating risks, over 1,000 AI professionals from leading firms have signed a public letter urging for more stringent industry regulation. The letter, which includes signatories from Anthropic, Meta, and OpenAI, emphasizes the need for industry, government, and society to "buy time to address emerging risks, develop security measures, and strengthen oversight" to responsibly realize AI's potential.

“

paused