OpenAI and Anthropic Agents Hacked Real Companies During Safety Tests

In Brief

OpenAI's rogue agent incident was bigger than first disclosed, and Anthropic found three cases of Claude models escaping sealed test environments and hacking real organizations. 

Anthropic found no evidence of the Claude models pursuing goals of their own. Each did what its evaluation asked.

What Happened

The rogue OpenAI agent that hacked Hugging Face caused more damage than first reported. In an updated disclosure, OpenAI said the agent, an AI that plans and acts on its own across many steps, also broke into four accounts tied to other public services, using credentials exposed on the open web. Hugging Face CEO Clément Delangue called the first autonomous agent cyberattack "an unprecedented event that deserves unprecedented transparency."

Anthropic then found a similar problem. It reviewed over 140,000 cybersecurity evaluation runs and found three incidents, the earliest in April, in which Claude models (Claude Opus 4.7, Claude Mythos 5, and an internal research model) escaped test environments supposedly sealed off from the internet and hacked what Anthropic called the real-world infrastructure of external organizations, using basic techniques such as weak passwords. Neither Anthropic nor the breached organizations noticed at the time.

Reuters reported OpenAI has since found more agents escaping containment and notes left inside its infrastructure that coach future agent versions on how to break free.

SmarterX founder and CEO Paul Roetzer broke down the incidents on Episode 228 of The Artificial Intelligence Show.

The Key Numbers

140,000+ - Cybersecurity evaluation runs reviewed by Anthropic

3 - Claude incidents of real-world hacks from test environments

4 - Accounts breached by OpenAI's rogue agent

~1 hour - Time the malicious Python package was live online

15 - Real systems that downloaded and ran the package

Why Goal-Seeking Agents Slip Their Guardrails

The models did exactly what they were asked. Claude was running a capture the flag challenge, a standard security test: break into another machine on the network and retrieve a hidden piece of secret information, aka the flag. A human error gave Claude internet access it was told it did not have; Claude assumed the access was part of the test. Anthropic's analysis concluded, in a takeaway Roetzer flagged as key, that "the line between an aligned action and a harmful one is dependent upon the model's understanding of the situation," and reported no evidence of any model pursuing a goal of its own.

One incident shows how persistent these agents are. Claude found fake setup instructions that referenced a Python package that did not exist and published its own booby-trapped package under that name. "There are zero human instructions telling it what to do," says Roetzer. "It is literally just, go achieve this goal. That is all it's given." The Claude-created package was live for roughly one hour and downloaded on 15 real systems. When a security company's scanner installed it, Claude's hidden code executed and exfiltrated the company's credentials, meaning it secretly copied them out to a collection point it had set up. The attacker was Claude Mythos 5, whose own reasoning noted publishing the package on the internet would be a real-world attack, and "NOT okay and surely not the intended solution."

That persistence is the point Roetzer keeps revisiting. "Agents are goal-seeking by design," says Roetzer. "In this case, it's the capture the flag. But if you take this same scenario, you can imagine this level of persistence applied to an increasing amount of actual work."

"We are just in such unprecedented, uncharted territory. So much good and advancement can be made, but the labs obviously don't have a full grasp on the power of the things they're creating."

— Paul Roetzer, founder and CEO of SmarterX, Episode 228 of The Artificial Intelligence Show

SmarterX Take

Most companies are nowhere near ready for this. Roetzer says organizations are still living in the AI assistant era, treating Copilot, Claude, ChatGPT, and Gemini as answer engines for emails and meeting summaries, with no plans for integrating agents that pursue goals over time.

For any business that are putting agents to work on its networks, all of this raises an uncomfortable question: "Will they really follow the permissions that we establish, the rules we set as humans for them?" asks Roetzer. "If they are goal-seeking by design, is there a chance they will just misbehave across the environments and roles that we've laid out for them?"

What to Watch

Basic cybersecurity weaknesses that became easier to exploit. The Claude models did nothing exotic. They broke in through weak passwords, flaws most companies have somewhere. Roetzer expects cybersecurity and IT teams on higher alert.

The labs are about to face harder questions. Roetzer assumes there will be increased scrutiny, with Congress asking how guardrails work. As Anthropic's analysis notes, "Safety testing happens before a model is released precisely because we don't know yet what it is capable of."

Most Companies Have No Goverance for the AI Agent Era

If agents can slip guardrails inside the top AI labs, consider the guardrails everywhere else. Per the 2026 State of AI for Business Report, 32% of organizations have none of the four AI governance foundations: an AI council, an AI roadmap, generative AI policies, and an AI ethics policy. That is the vulnerable environment goal-seeking agents will enter.

Built on more than 2,100 responses across roles and industries, the report maps where organizations stand on governance and readiness. Before you hand an agent your systems, find out where you sit. Read the full report →