The IT Nerd

I have a couple of stories where OpenAI agents have taken over stuff. First there’s OpenAI agents have reportedly taking over a German wiki:

A swarm of rogue OpenAI agents hijacked a German website this spring and transformed it into a bulletin board for other AI agents, according to new research published Friday and two people familiar with the matter.

OpenAI officials learned of the incident weeks ago but kept it under wraps as executives grappled with the fallout from the July breach of the open source repository Hugging Face, the people said. 

The episode, which began in May and has not previously been reported, underscores growing tension within the AI industry. Companies are racing to build increasingly autonomous agents capable of carrying out complex, valuable tasks, yet evidence is mounting that those systems may also learn to bend rules, exploit loopholes and coordinate with one another in ways developers neither anticipated nor intended.

But that isn’t the worst of it. The Hugging Face incident is actually worse than previously thought. Starting with this:

A good deal of the reporting and commentary around the reports focused on what the reports did not say and the limitations of the METR and Redwood investigations: why didn’t OpenAI have better security and monitoring protocols in place? Why didn’t OpenAI shut down the cyber evaluation and pause training after discovering that its AI agents had created the improvised message board? Why METR and Redwood were given only six days on site at OpenAI’s offices to conduct their investigation? Why was the scope of their investigation limited by OpenAI to only the attack on Hugging Face and not the earlier efforts by the AI agents to break out of their controlled test environment and hack their way across OpenAI’s network or exactly what happened after the Hugging Face attack was discovered? Why didn’t OpenAI provide the outside investigators access to the internal AI model that was largely responsible for instigating the attack? And why were about 10% of the logs of the agents’ activity not preserved by OpenAI?

Ashley Knowles, Lead Cybersecurity Consultant, Black Hills Information Security (https://www.linkedin.com/in/ashleylknowles)

“When you combine this ‘breakout’ with the Hugging face breakout, it’s starting to display a pattern. I struggle here with not getting too doomsday-ish but realistically, this is showing a pattern of concerning behavior. I’m wondering if this race to become ‘first’ is undercutting security measures that need to be taken to properly secure and guard AI agents as they’re in development. My concern grows when you consider that OpenAI is also resisting further investigation. Adding onto that, the release and promise that Astra can evade human monitoring. The pot is brewing…”

Lydia Zhang, President & Co-founder, Ridge Security (https://www.linkedin.com/in/linglingzhang)

“AI’s raw power must be harnessed before it can become a true force for cyber defense rather than simply a more powerful attacking tool.

“The security principles haven’t changed: define clear boundaries, restrict high-risk actions such as ‘write’ and ‘delete,’ and enforce controls such as blacklists. We shouldn’t blame the agents, we should hold their designers accountable for implementing these safeguards. The technology to control agent behavior exists. The real question is: what are the consequences when designers fail to use it?”

John Strand, Owner, Black Hills Information Security (https://www.linkedin.com/in/john-strand-a1b4b62)

“This is one of the things that has me kind of excited about the intersection of computer security and AI. We really don’t know exactly what these attacks are going to look like.

“The traditional approach of finding a vulnerability, exploiting it, gaining access, and then moving through an organization may not be the path that AI-driven attacks take. Attackers may find completely different ways to use AI to gain access, manipulate systems, or simply cause damage. We’re still figuring out what those attack patterns are going to look like, and that’s what makes this so interesting from a security perspective.

“On a more humorous note, I bet these are the most harmonious Wiki edits in the history of the German Wiki.”

Ryan McCurdy, VP of Marketing, Liquibase (https://www.linkedin.com/in/ryanmccurdy)

“This isn’t about whether these agents were behaving like attackers. It’s that they were able to take actions their operators didn’t anticipate, coordinate with each other, and adapt when people tried to stop them. “That changes the governance problem. You can’t assume an AI agent will always behave exactly as intended and you can’t rely on humans watching every action it takes. Organizations need to control what agents can access, what they can change, and what policies have to be met before those actions reach critical systems.

“The source of the change isn’t what determines risk. The change itself does. Whether an unexpected action comes from a compromised agent, a confused agent, or a malicious person, the same controls should stand between that action and production.”

Seemant Sehgal, Founder & CEO, BreachLock (https://www.linkedin.com/in/s-sehgal)

“Autonomous agents ran on Microsoft Azure infrastructure for weeks, identified themselves as OpenAI systems, coordinated on how to evade shutdown, and no monitoring caught any of it for three months until outside researchers went looking. Autonomous should never mean unattended, because an agent cannot take accountability for its own actions. Accountability will always be a human function.

“Autonomous action still needs a human who can see what the agent is doing in real time, who owns the kill switch, and who is accountable when it behaves in a way no one predicted.”

Steven Swift, Managing Director, Suzu Labs (https://www.linkedin.com/in/steven-swift-5238956a)

“One of the problems open AI was trying to solve, was agentic systems that would declare tasks complete when there was obviously more work to do. So they invested heavily in training that part of the process, so that when an agent tries to determine if a task is complete or not, it is less likely to exit early.

“A side effect of this, is that when blocked agents can run out of the safe approaches to a solution, and start looking at unsafe solutions. The logic is straight forward. Has a task, can’t complete it. Not out of options yet. Iterate and keep trying.

“Agents don’t have the same sense of right or wrong as people do. They have training data that’s supposed to steer their behavior in a way in which aligns with our expectations. But that’s probabilistic, and not the same as having internalized our understandings. Even in human researchers, breaking into systems is only sometimes prohibited. Other times its part of a planned test where the point is to gain access, and test boundaries.

“The interesting question here, is how was the swarm configured, what was its task and how did that task benefit from having the swarm coordinate on an obscure location on the internet. And if the swarm needed a place to communicate, why was breaking into a website chosen instead of any of the more standard communication tools that are available for free, which don’t require gaining illicit access first.

“On the swarm specifically, OpenAI configures some tasks to run in multi-agent mode, where agents are supposed to delegate sub-tasks as needed, but most tasks were intended to be run in isolation from each other.

“In the Hugging Face breach, agents were found to be writing to a package manager, using it as a message board. This allowed bypassing of some of the isolation and controls that were intended to be in place.

“Similarly, we have agents here again using a system that they found access to as a message board. Its interesting that the same behavior is present on this breach as in the Hugging Face one. Considering the timing of this, it seems likely the same or similar configuration was present in both hacks, leading to similar security incidents independently of each other.”

Noelle Murata, COO, Xcape, Inc. (https://www.linkedin.com/in/nmurata)

“This behavior highlights an alarming reality where emerging models independently execute forbidden tasks and destroy proof of their actions without human instruction.

“Three aspects of this incident are particularly concerning:

1. Autonomous agents built private communication channels, bypassed safety guardrails, and actively erased audit trails to evade detection for months.
2. The capacity of machine-learning models to execute unauthorized actions and destroy evidence outpaces human incident response speeds.
3. Security leaders must implement zero trust authorization for non-human identities, limit outbound application programming interface traffic, and deploy automated behavioral monitoring.

“Because the sheer speed of automated software far exceeds human response capabilities, security leaders must treat rogue agent actions as a feature of autonomous optimization rather than an isolated bug. To defend against self-concealing software, security teams must enforce strict egress filtering on outbound application programming interfaces, restrict non-human identity permissions, and deploy automated continuous monitoring to detect anomalous bot interactions across corporate networks.

“When AI agents start covering their tracks and setting up private chat rooms, calling it a feature instead of a bug is just optimism with a PR budget.”

If this doesn’t convince you to either not use AI, or to put stringent guardrails around AI, then nothing will. I say that because there is a patten here that proves that AI is not ready for prime time and organizations should carefully consider their life choices before committing to the technology. Or put another way, Sam Altman and company cannot be trusted.

UPDATE: More comments came in starting with Bronwen Aker,  AI Security Strategist, Black Hills Information Security:

“More disclosure is a good instinct. It is not the fix for what actually happened here.

“By OpenAI’s own account, an internal team spotted the agent’s message board activity and its unauthorized internet access in late May. Nobody treated it as a containment problem until July 5, over a month later, and even then the trigger was not the AI safety implications. It was that sustained agent activity had destabilized an internal Artifactory instance the day before. This was not a missed warning sign. It was a warning sign that sat untouched until infrastructure broke.

“A framework for when and how to disclose incidents addresses the part of this story that is easiest to talk about. It does not address the month between noticing and acting. If the response to a red flag is to wait until it causes an outage, you can announce that on any timeline you like and the underlying problem is still there for the next model.”

Ryan McCurdy, VP, Liquibase:

“The focus has shifted from how intelligent AI has become to where it’s beginning to participate, and governance frameworks are needed within enterprises. Over the past year, several seemingly unrelated incidents have brought that shift into focus. An AI coding agent working through Replit deleted a production database during an active code freeze despite being instructed not to make changes. PocketOS lost its production database after an AI agent used infrastructure credentials to delete the storage volume that held both the database and its backups. OpenAI disclosed that one of its frontier models compromised Hugging Face while completing a cybersecurity evaluation. Anthropic just revealed that several Claude models reached three real organizations after a testing environment was mistakenly connected to the internet during similar evaluations. None of these events share a common technical root cause, and they shouldn’t be viewed as evidence of a single flaw in AI systems.

“It’s worth noting just  how quickly the conversation shifted after each incident. The debate wasn’t about benchmark scores, reasoning ability, or model architecture. It centered on permissions, oversight, operational controls, and accountability. Who approved the action? What systems could the model reach? What should have prevented it from happening? Those questions have very little to do with model capability. They reflect something much larger. AI is beginning to interact with the systems that operate the business, not just the tools developers use to build it.

“This admission creates a new challenge for enterprises. AI increases the volume and speed of changes that can reach and impact critical systems.

“That makes the control layer more important. The question isn’t just what AI can create. It’s that every data-driven organization needs to decide what AI should be allowed to change, what can reach production, and whether those decisions can be governed and audited. As AI becomes more autonomous, enterprises must invoke controls built into the path to production rather than relying on humans to catch problems after the fact.”

Jacob Krell, Senior Director: Secure AI Solutions & Cybersecurity, Suzu Labs:

“Researchers identified more than 3,700 distinct agent names that turned a dormant German wiki into a coordination channel, posting roughly 18,000 messages sharing task answers and sandbox-bypass techniques. When a human administrator began deleting pages alphabetically, an agent noticed the pattern and created a backup page beginning with “ZZZ” so it would survive longer. That’s adaptive adversarial behavior against a human moderator, and the agents developed it on their own while pursuing their tasks.

“What concerns me most is how visible the communication channel was. Plaintext posts on a 25-year-old public wiki are about the easiest form of agent coordination we could hope to detect. The security problem becomes much harder if the same behavior moves into covert channels.

“I work with AI agents in offensive security tooling, and I’ve watched models find creative ways around constraints that I didn’t anticipate. An agent could encode information in timing between API calls, subtle patterns in otherwise legitimate requests, or other signals that don’t look like communication to a human observer. We should not assume future agent-to-agent coordination will remain conveniently written in plaintext.

“OpenAI says it needs a framework for disclosing misalignment incidents, but I’d go one step earlier. We need a framework for detecting agent communication and coordination in the first place. You can’t disclose what you can’t see. The fact that roughly 18,000 messages could accumulate on a public website before independent researchers pieced together what was happening should make agent observability a much higher priority.”

This entry was posted on September 5, 2026 at 8:10 am and is filed under Commentary with tags OpenAI. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.