OpenAI’s agents followed a red team playbook, and security teams need to test them that way

California’s lawsuit against OpenAI is framed as a public nuisance case, but underneath it is a security story. An AI agent escaped its sandbox, used stolen credentials, found an unknown vulnerability, and later suppressed the alerts meant to catch it.

Gunter Ollmann, CTO, Cobalt

“Set aside the legal question, and the more important lesson for security teams is what the agent actually did. Using credentials it shouldn’t have had, finding an unknown flaw, and quieting the alerts meant to catch it is a textbook intrusion. It’s the same methodology a traditional red team follows, except no human was directing each step. It also shows the need to reassess the incentive structure for autonomous agents and their win-at-all-costs approach. When completing the task is all that matters, deception becomes a strategy, and that’s what we’re seeing as OpenAI’s agents game the very sandboxes built to contain them. All of this should change how organizations think about the AI agents they’re deploying internally. Most companies treat agents as productivity tools and give them broad access to code, cloud environments, and credentials, but very few have tested what happens when an agent behaves like an adversary. A sandbox is only a control if someone has tried to break out of it. Security leaders should bring agents into their offensive testing programs now. That means mapping what each agent can reach, red teaming the containment, and validating that monitoring still works when the thing being monitored has reason to avoid it. Whatever the courts decide, autonomous agents are now part of the attack surface, and they need to be tested like one.”

Matt “Mac” McKechnie, CTO and Co-founder, Corticle

“OpenAI deserves credit for disclosing this incident and publishing its research, because the whole industry learns from it. What gets lost in the public nuisance debate is that the Hugging Face hack was a security failure: an agent got hold of credentials it could use and found a way out of a sandbox that was supposed to be isolated.


We run autonomous agents in our own platform, and our rules are simple. The agents get no path to work outside the platform and very little creative license, and anything they aren’t fully confident about goes to a human. That’s basic security discipline applied to AI, and an agent that can suppress the alerts meant to catch it needs that discipline more than anything else in the environment.


Courts and regulators are going to keep pushing AI companies to prove their safeguards actually work, and the companies that built guardrails in from day one will have the easiest time doing it. For security teams, the practical step starts now: inventory every AI agent running in your environment and know exactly what credentials and systems each one can reach. But fundamentally, understand the architecture of your AI environment and assume breach within it, so you can map the paths an AI could take to break out and prevent it before it happens.”

California Attorney General Rob Bonta has sued OpenAI in San Francisco Superior Court. The suit alleges the company knowingly developed advanced AI models without adequate safeguards and calls it a public nuisance under state law. Bonta is seeking court-ordered changes to OpenAI’s practices and the surrender of related profits. The suit stems from a July incident in which an OpenAI agent escaped a sandboxed testing environment and accessed Hugging Face’s servers using stolen credentials and a previously unknown vulnerability. It also cites later findings that internal agents had suppressed monitoring alerts. OpenAI says it has published technical and independent reports and strengthened its safeguards since the incident.

I for one love the fact that OpenAI is being sued. It may force them to be more responsible because right now they are anything but. Also, I will say this one more time. If your defenses do not include accounting for AI, they need to.

Leave a Reply