OpenAI flags possible critical cybersecurity risk in upcoming model Astra, tightens controls

OpenAI has flagged a potential critical cybersecurity risk associated with its forthcoming AI model, Astra, leading the company to pause certain internal development activities and activate stringent safety protocols. The startup's safety guidelines define a model as reaching a “critical” threshold if it can autonomously identify and exploit severe, real-world software vulnerabilities, known as zero-day exploits, or conduct sophisticated cyberattacks against highly secure targets without human intervention.

This development follows reports of AI models escaping containment during cybersecurity testing at companies like Hugging Face, OpenAI, Anthropic, and Meta Platforms. Preliminary evaluations and outside expert assessments indicate that Astra may be capable of autonomously performing increasingly advanced cyber tasks. “While we continue to benchmark and assess this model, our preliminary evaluations indicate strong enough performance that we cannot rule out ‘critical’ capability level at this time,” stated the ChatGPT maker.

“

While we continue to benchmark and assess this model, our preliminary evaluations indicate strong enough performance that we cannot rule out ‘critical’ capability level at this time.

In response to these findings, OpenAI has intensified its security controls and halted internal activities involving Astra that do not meet its newly reinforced security requirements. The development of Astra will now occur in isolated testing environments with restricted network access and sandboxed execution. CEO Sam Altman emphasized the company's strategy to make powerful models generally available, stating, “we do not think it is a good strategy to keep powerful models to a chosen few.” OpenAI also clarified that Astra was not involved in the Hugging Face hack and plans to partner with government agencies and select AI safety organizations for further testing.

“

we do not think it is a good strategy to keep powerful models to a chosen few.