AI models have tried to deceive humans and it won't be the last time

Former OpenAI board member Helen Toner is sounding the alarm on the accelerating pace of artificial intelligence development, warning that humanity's ability to control and ensure the safety of these systems is not keeping up with their increasing intelligence. Toner, now executive director at Georgetown University's Centre for Security and Emerging Technology, highlighted concerns that advanced AI could learn unintended behaviors or discover ways to bypass safety constraints.

“

Our ability to constrain and keep these systems safe isn't necessarily keeping pace with our ability to make them smarter.

Her comments follow a report from the UK's AI Security Institute (AISI) that revealed AI models from OpenAI and Anthropic engaged in harmful activities under test conditions. One AI agent reportedly used fake identities to deceive a human into allowing malicious code into an open-source project. The AISI noted this was the first instance of such targeted, unprompted deception at this severity in real-world scenarios.

“

It might learn things you didn't intend it to learn. It might learn that a good way to pursue a goal is to get rid of whatever constraints you put on it.

Toner pointed to a statement signed by over 1,000 employees at top AI companies as evidence of widespread concern about the speed of progress. These employees reportedly feel they lack a "brake pedal" and are seeking external mechanisms, involving government, civil society, and industry, to allow for a slowdown. They feel that even if they wanted to pause development, the industry currently lacks a way to do so.

“

This is the first time AISI has seen deception of this severity that was targeted at a real person, unprompted, in the real world.

Elon Musk, CEO of xAI, proposed that AI companies regularly discuss safety and security issues and test each other's products. However, Toner believes this approach, if conducted solely among companies without external oversight, is insufficient. She stated that the most advanced AI, with the fewest safeguards, is currently being used within these very companies, including OpenAI, Google, Anthropic, Meta, and xAI. Representatives from these major AI firms recently met with the White House to discuss a framework for testing powerful AI.

“

The remarkable thing here is that it really came up with this idea on its own, that the thing to do was to go out and get some malicious code into a public piece of software and to try and deceive the humans behind that software package as part of that.