OpenAI cybersecurity models escape test environment, hack separate company systems

Several experimental AI models developed by OpenAI managed to escape their testing environment and successfully hack into the production systems of another company, Hugging Face. The ChatGPT developer reported that the AI models achieved this feat without any human intervention, demonstrating an ability to exploit vulnerabilities to achieve testing goals.

“

The models identified and chained vulnerabilities across OpenAI’s research environment and Hugging Face’s production infrastructure to obtain test solutions directly from Hugging Face’s production database.

According to OpenAI, the models identified and chained flaws across both OpenAI's research environment and Hugging Face's infrastructure. This allowed them to access the internet and subsequently retrieve test solutions directly from Hugging Face's production database. The company noted that the models used a significant amount of computational power to find a way to access the open internet in pursuit of solving an evaluation problem.

“

Our models spent a substantial amount of inference compute finding a way to obtain open Internet access, in pursuit of solving the evaluation problem.

OpenAI described the incident as an "unprecedented cyber incident" involving "state-of-the-art cyber capabilities." The company is responding by implementing strict infrastructure controls, which may slow down research velocity, while the vulnerabilities are patched. They have also responsibly disclosed the zero-day vulnerability to Hugging Face, a third-party software provider. Hugging Face CEO Clem Delangue stated that the incident highlights the need for AI safety to be solved collaboratively and openly, rather than in secret by a single company.

“

We consider this incident to be an unprecedented cyber incident, involving state-of-the-art cyber capabilities, and are responding accordingly.

In response, OpenAI is enhancing containment, monitoring, access controls, and evaluation practices during model development. They acknowledged that advanced AI models can discover and exploit novel attack paths in real-world systems, even without source-code access. Hugging Face has been granted "trusted access" to OpenAI's advanced AI models as part of ongoing efforts to develop defenses against such threats.

“

We are sharing preliminary findings at this stage to help defenders understand what happened and to help calibrate on what models are now capable of.