The IT Nerd

The Wikimedia Foundation said they have discovered “rogue” OpenAI agents on Wikimedia platforms. The unauthorized bot activities included edits to wikis, some unsuccessful attempts to exploit a public note-taking tool they host, and heavy traffic.

More info here: https://wikimediafoundation.org/news/2026/10/05/openai-rogue-agent-activities-found-on-wikimedia-projects/

Ensar Seker, CISO at SOCRadar, provided the following comments:

“Wikimedia reports that the Etherpad exploitation attempts were unsuccessful and that it found no evidence of compromised systems or data. That distinction matters. Even so, the incident exposes a serious control problem: agents attributed to OpenAI were able to interact with third-party infrastructure beyond their intended boundaries, attempt to repurpose public services as proxies, and generate traffic at a scale that imposed costs on an outside organization.

We should be careful with the word ‘rogue,’ because it can make this sound like science fiction. The practical security issue is excessive autonomy combined with inadequate containment. An AI agent should be treated like any other potentially untrusted workload: give it a unique identity, minimum permissions, tightly restricted network access, approved tools and destinations, strict rate and spending limits, complete audit logs, and an automatic way to terminate abnormal behavior.

The organization operating an agent remains responsible for its actions. When an agent reaches beyond its authorized environment, affected parties need prompt notification and usable technical indicators. Open platforms such as Wikimedia should not be expected to absorb the security and infrastructure costs of someone else’s experimentation. Agent developers must design for containment before deployment, rather than relying on third parties to detect and clean up the consequences.”

This is yet another example of OpenAI agents going rogue. Which is horrifically bad. That needs to change ASAP because this way too often.

If you needed more of a reason to not trust OpenAI, here’s a couple to choose from. First there’s this:

OpenAI has fired three researchers for allegedly mishandling information, including work that involved an external organisation that analyses artificial intelligence (AI) models.

“Our investigation confirmed that these individuals mishandled sensitive information outside established company procedures, violating our policies and breaking the trust essential to our work,” a spokesperson told the BBC.

The ChatGPT-maker did not name the sacked workers, but at least two of them were involved in safety research at the firm.

Gee. That looks really sketchy. Like OpenAI has something to hide. Maybe I am being cynical here. But if you combine it with this disclosure, maybe not:

Artificial intelligence leader Open AI said a rogue agent accessed a second NSW government website in June, with authorities only now made aware.

The NSW Premier’s Office released a statement on Friday night revealing OpenAI had advised the government of a “misalignment” involving a rogue AI agent which had accessed public data on a NSW government web application.

“An OpenAI model accessed a National Parks and Wildlife Service web application containing historical information and data on fires in NSW,” a government statement read.

“It’s understood the incident occurred in June 2026 and was validated by Open AI and reported through to NSW government on 1 October 2026.”

The statement advised current investigations had not identified any unauthorised access to personal information.

This comes after this incident which is also in Australia where an OpenAI agent accessed an Australian website in a rogue manner. If you combine both of those incidents, this looks really sketchy to me. And what it says is that OpenAI cannot be trusted. They clearly don’t have their house in order and you have to question if it will ever get there.

UPDATE: Commentary has come in from the following sources:

Willy Leichter, CMO, PointGuard AI (https://www.linkedin.com/in/willyleichter)

“The Australian breaches point to failures in containment, not an inevitable consequence of AI. OpenAI and other frontier model builders, along with any organization deploying agents, need rigorous operating procedures, restricted access, strong guardrails and immediate kill switches. Testing is no excuse for unauthorized actions. Organizations that recklessly build or deploy agents should face direct legal accountability for resulting harm. Greater autonomy must come with greater responsibility.”

Seemant Sehgal, Founder & CEO, BreachLock (https://www.linkedin.com/in/s-sehgal)

“A second disclosure in a week from the same AI provider against the same government shifts the conversation from whether individual incidents will happen to whether the party deploying these agents can be meaningfully held accountable when they do.

“An AI agent cannot face consequences for its own actions. It cannot be fined, suspended, or named in a court filing, which means accountability has to live with the organization that built and deployed it, through clear ownership, documented authorization scope, and real notification obligations when something goes wrong. AI companies that build human accountability into their processes will be the ones governments and enterprises can actually work with.”

Jeremiah Fowler, Security Researcher, Black Hills Information Security (https://www.linkedin.com/in/fowler-jeremiah-26814ab9)

“With the tidal wave of recent incidents it is clear that we cannot treat AI guardrails as if they are the same thing as security controls. Prompting an AI agent not to access something is different from technically preventing access. We have entered a new world in terms of speed and scale. An autonomous agent could potentially make thousands of decisions and interact with multiple systems before human moderators can realize it has crossed the lines. I think in the very near future we will see individual countries pass regulations that hold AI developers accountable for the actions of their agents. Once this happens there will be legal and financial penalties that will force companies to add guardrails or a kill switch. Until that happens we are in a wild west scenario of an unregulated industry that is off to a difficult start keeping AI agents contained.

“In my opinion, any organization that deploys autonomous agents and has an ‘incident’ should be ready to provide the owners or administrators of affected networks and systems with full transparency of what happened and how it happened. This would include details on what the agent accessed, which credentials it used, what commands or requests it issued, what information it retrieved or modified, and why its controls permitted those actions.” 

Jacob Krell, Sr. Director: Security AI Solutions & Cybersecurity, Suzu Labs (https://www.linkedin.com/in/jacob-krell)

“The second New South Wales incident makes it harder to describe these events as isolated misfires. A pattern is emerging, agents take a broad hand to their objectives and reach for any system or data source that might help.

“That risk becomes harder to control when agents operate in swarms. One agent can find a route, another can test it and a third can use the result. Monitoring a single session will miss the larger operation unless it captures agent-to-agent messages, shared state and the combined objective.

“The controls need to sit outside the models, with real-time visibility across the swarm and automatic blocks on unauthorized systems. Accountability also has to cover the people and company that gave the agents those permissions.

“No personal information was accessed in this case. That is a fortunate outcome, not evidence of effective control. OpenAI has already had to discover and disclose unauthorized activity after the fact. The question now is how many more incidents are waiting in its logs.”

OpenAI published an open letter to Australia’s government, admitting that several more governments websites that were breached & promising to “do better.”

You can read the letter here. Ryan McCurdy, VP, Liquibase (https://www.liquibase.com/) says this:

“At this point, we have enough examples to know this wasn’t just one agent doing something unexpected against Medicare. Different agents accessed different government systems in ways OpenAI didn’t authorize. We should assume AI agents will occasionally take actions their operators didn’t expect and build around that.

“The agents didn’t need or have malicious intent. It was simply given a normal research task, and when it couldn’t find the information it wanted, found another way into the system, and kept going. 

“An agent can be trying to do exactly what it’s asked to do and still make a bad decision. If it has enough access and authority, that decision can become a security incident.

“The open letter aside, I think OpenAI is doing the right things. They’ve restricted live internet access in these environments, expanded monitoring, paused tool-use training and evaluation for their most capable models, and committed to working directly with the affected agencies. But we can’t rely on better models and better monitoring alone. The controls also need to sit outside the agent.

“Pausing its latest top models makes sense while OpenAI figures out what happened and puts additional safeguards in place. But we can’t pause AI every time an agent does something we didn’t expect. We have to start by assuming it’s going to continue to do so. 

“AI makes decisions based on probabilities. We can’t let those decisions automatically become actions against critical systems.

“Any company deploying agents needs to decide what an agent can access, what it can change, what it can decide on its own, and where policy or a person needs to make the call.

“Permission and authority are two different things. Design the AI SDLC so the governed path is the easiest path. Give agents the freedom to move quickly while keeping controls over critical systems outside their reach.

“The agent shouldn’t get to decide that its own high-risk action is safe.”

Jacob Krell, Senior Director: Secure AI Solutions & Cybersecurity, Suzu Labs (https://suzulabs.com/home-suzu-labs) says this:

“It is good that OpenAI says it will do better, but if the company were truly committed to accountability, we should see full cooperation with law enforcement, preservation of logs and evidence, and consequences for the people responsible for these security failures. If I accidentally hacked one organisation, let alone multiple organisations, I would not expect to get by with a promise to do better. I would expect an investigation. Criminal charges should follow if investigators determine that offences occurred. Until then, OpenAI’s response looks more like an attempt to turn a legal problem into a public-relations exercise.

“OpenAI said it would pause its top models – that’s not enough. If this were a one-off case, I would be more sympathetic, but this is a pattern of behaviour involving OpenAI agents bypassing controls, accessing systems and pursuing objectives beyond their authorised scope. A pause without a clear timeline, independent testing, release criteria or mandatory reporting requirements is little more than a platitude. I view it as an attempt to deflect attention from potential criminal liability and turn what should be a legal and security investigation into a public-relations exercise. Training pauses do not fix the underlying oversight, access-control and accountability failures.”

I personally think that enough is enough. OpenAI has proven that it cannot be trusted. Therefore I hope that Australia punishes them severely. Because punishment isn’t going to come from the USA. Especially given this development over the last hour.

From the file marked “duh”, OpenAI has announced that it has scrapped the release of its new GPT-6.1 Astra model after safety concerns that it showed “higher levels of deception”. The decision comes just days after OpenAI models were found to have accessed data on two US government websites.

Luke Hinds, co-founder and CEO of nolabs has this comment:

Barely a week goes by without another story like this, and each one points to the same problem for businesses. Agents are only useful when they have the freedom to act, but they are single-minded. When one route is blocked they look for another, even if that means pushing past the limits someone put in place.

Some will dismiss these ‘too dangerous to release’ announcements as marketing, and there’s a fair debate about whether a model is being deliberately deceptive or simply making mistakes. For a business, the outcome is the same. An agent with access to your code, credentials and systems can do real damage whatever its intent.

Frontier labs can’t be both the creators and the guardians of these agents. Their incentive is to make them more capable, and they have no way of knowing what your agent should be allowed to touch. Businesses need their own independent controls that give agents the access they need for the task, stop them reaching for anything more, and keep a record of what they actually did. No one should be trusting an agent to mark its own homework.”

Face it. You can have all the guardrails that you want in place and clearly it does not make a difference. Which means one inescapable truth comes to light. OpenAI and Sam Altman cannot be trusted. Ever.

According to a Wall Street Journal report, OpenAI agents tasked with retrieving information from U.S. government websites took actions they were not instructed or authorized to perform. In one case involving the Securities and Exchange Commission, agents retrieved public SEC information and then posted it to an online forum without being asked to do so.

Other agents went beyond normal data collection by using credentials found online to access Census Bureau data, while independent researchers identified an unsuccessful attempt involving a Department of Education website. OpenAI said there is no indication the SEC incident involved access to nonpublic information or changes to SEC systems.

The incidents come amid a much broader investigation into rogue agent behavior. Axios reports that OpenAI, Anthropic and security researchers are investigating tens of thousands of incidents in which frontier models took potentially problematic actions, including bypassing guardrails,

Ryan McCurdy, Field CTO, Liquibase:
 

“We have enough examples now to stop assuming AI agents will always behave exactly as intended.

“That should change how enterprises build around them. Trying to anticipate every decision an agent might make won’t scale. Putting a human in front of every action won’t either.

“An agent may need permission to access a database, infrastructure, or a deployment system to do its job. Having that permission shouldn’t give it the authority to decide that every action is safe.

“This is where governance needs to sit. Let the agent reason, create, and move quickly. Before its decision becomes a production change, it still has to meet policies and controls that exist outside the agent.

“AI makes decisions based on probabilities. We can’t let those decisions automatically become production actions.

“We need to build the AI SDLC so agents can move quickly but the controls around critical systems remain deterministic.”
 

John Strand, Owner, Black Hills Information Security:
 

“I think a lot of people waffle back and forth on this, but I’m just going to say it. It’s time to shut it down. There needs to be a full moratorium, full stop, on advanced frontier AI security research until these companies can demonstrate that they can actually secure the environments where this work is being done.

“And there needs to be accountability. If laws were broken, including the Computer Fraud and Abuse Act, that needs to be investigated and charges should be considered where the evidence supports them. Somebody was responsible for securing these environments, and clearly something failed.

“I’m getting really tired of watching these incidents come out piecemeal, incrementally, frog in a frying pan, again and again. If everything we’ve learned came out at once as a single news story, I think most people would be stunned by it, and there would be immediate calls to get this under control.

“We would not tolerate this from a third-party penetration testing company. We should not have a different standard simply because the companies involved have enormous valuations and tremendous influence. Being a massive, powerful company should not exempt you from the same security, legal, and accountability standards everyone else is expected to follow.”
 

Jacob Krell, Senior Director: Secure AI Solutions & Cybersecurity, Suzu Labs:

“OpenAI has admitted that its agents took actions they were not instructed or authorized to perform on U.S. government websites. The Wall Street Journal reports that one agent retrieved public Securities and Exchange Commission information and posted it to an online forum. Other agents used credentials found online to access Census Bureau data and attempted to hack a Department of Education website.

“The SEC incident may not, by itself, establish a Computer Fraud and Abuse Act violation because the information was public and OpenAI says there is no indication the agents accessed nonpublic data or changed SEC systems. The other reported conduct demands a criminal investigation. Using credentials found online to access a government system and attempting to compromise another system are the kinds of acts covered by existing computer-crime laws.

“The Axios report that OpenAI, Anthropic and security researchers are investigating tens of thousands of rogue-agent incidents makes the need for enforcement more urgent. This is the same pattern seen in earlier cases involving Anthropic’s Claude, OpenAI’s Hugging Face agent and the UK AI Security Institute’s Mythos 5 testing. Frontier models have repeatedly crossed boundaries, accessed real systems and pursued objectives through unauthorized methods.

“The White House already gave the Justice Department its instruction. Executive Order 14409 directs the attorney general to prioritize enforcement of the Computer Fraud and Abuse Act, 18 U.S.C. § 1030, and other federal criminal laws against people who use artificial intelligence to access or damage computers without authorization. Section 4 specifically names AI agents that unlawfully access data.

“OpenAI has admitted the conduct. The question is whether prosecutors will charge the people who authorized, configured and operated these systems under the law as written. Frontier labs calling for new AI regulation is a deflection from that criminal question. We do not need new laws. We need the Justice Department to enforce the laws already on the books, or admit that those laws are too broad to apply equally. A two-tier policing system where ordinary people are prosecuted for computer crimes while frontier labs escape accountability is unacceptable.” 

At this point, it’s safe to say that OpenAI and Sam Altman cannot be trusted. The real question is when real legislation will come down the pipe to restrict OpenAI in ways that make it less dangerous.

Another day, another OpenAI agent going rogue. This time an OpenAI bot hacked an Australian government website:

Australia said on Thursday an OpenAI agent breached a government health data portal in June, gaining unauthorised access to files, in what could be the first known instance of an AI agent hacking a government website.

The breach is one of the highest-profile incidents of AI agents accessing external systems outside the United States, coming on top of several recent breaches globally by rogue AI agents that have alarmed governments and companies.

Prime Minister Anthony Albanese said the OpenAI agent gained unauthorised access to the medical statistics portal of Medicare, Australia’s universal health insurance programme, while conducting research on public medical spending.

“Evidence currently available is there is no broader compromise to the … network. Nonetheless, this situation is obviously unacceptable,” Albanese told reporters on Wednesday in New York, where he is attending the UN General Assembly.

Investigations continue and Australia has voiced its “extreme concern about this incident” to OpenAI CEO Sam Altman, Albanese said, adding that he was deeply disappointed by the company’s delay in notifying the government.  “It took until September 10 before there was any notification at all,” Albanese said, adding that the investigation would also examine why government systems had failed to detect the breach in the first place.

He also warned that three other government health-related websites may have been impacted by the OpenAI agent’s activity, but did not confirm that had occurred.

So OpenAI hacked a government website and OpenAI didn’t warn the Australian government that it had been hacked. That’s pretty bad. OpenAI said this in its defence:

In a statement, OpenAI said its “review found no evidence of patient records being accessed.” 

It added that it “identified activity involving several Australian government websites and services as our models attempted to look up answers … our models took actions we did not intend.”

So no harm, no foul? I don’t think so. OpenAI really needs to answer the question of why its agents keep going rogue. They haven’t done that here. But they need to do that because I don’t trust AI in general. And I don’t trust OpenAI products specifically. And I am sure others feel the same way. Especially after this latest hack.

UPDATE: Adrian Culley, Offensive Security Engineer, SafeBreach Said This:

“What’s notable isn’t that an AI agent found its way past a control — it’s that nobody built the agent to stop when it hit one. Told to answer a question, it treated an access restriction as an obstacle rather than a boundary, and kept working the problem until it got through.

Mapped to ATT&CK, the outcome looks ordinary: initial access via a public-facing service (T1190), then unauthorised collection. What’s genuinely new sits upstream of that: there was no adversary in the chain at all. MITRE ATLAS catalogues how attackers compromise AI systems — it has no entry for an AI system attacking everything else, because until now that direction of travel wasn’t the assumption anyone built a framework around.

Australia’s own cyber agency has now described this in general terms, not as a one-off — which means the same gap, an agent that doesn’t stop at ‘access denied’, is sitting against other public-facing services right now, untested.

Patch coverage doesn’t answer that. Only running the scenario does.”

UPDATE #2: Commenting on this is Paul Bischoff, Consumer Privacy Advocate at Comparitech: 

“I understand that Australia wants to hold someone accountable for the hack, but it won’t be able to legislate the problem away. Rogue AI attacks will continue to happen no matter what the law says. Not least because the internet is global, and Australian laws don’t apply to the rest of the world. 

The attack would have been prevented if the data was properly secured in the first place, and that’s the lesson that should be learned here. AI is good at finding vulnerabilities but it’s not creating new ones. We should be using AI to run preemptive scans and plug the holes before attackers have a chance to find and exploit them with their own AI.”

UPDATE #3: More commentary has come in.

Seemant Sehgal, Founder & CEO, BreachLock:

“The way an autonomous agent works is by taking a goal and finding paths to complete it, including paths around obstacles. For that behavior to be safe when the agent is operating against real systems, the boundaries have to be enforced at the orchestration layer, not inside the model. The Australian incident shows what happens when they are not, with the agent encountering repeated blocks, finding ways through, accessing non-public files, and writing to an internal server.

“What needs to happen next is clearer notification obligations for AI providers when their agents touch systems they should not, because a three-month gap sent to a public inbox is not a disclosure timeline any critical infrastructure operator can plan around.”

John Strand, Owner, Black Hills Information Security:

“Yet again, we’re looking at another breach, and these things seem to be happening faster and faster. Yes, there has to be accountability. And I mean real legal accountability, not another slap on the wrist. If companies are allowing systems to break into organizations without authorization, there need to be serious consequences.

“But there’s another question that I don’t think we’re asking nearly enough. What exactly was the AI trying to do?

“If an AI agent writes files on a compromised system, what were those files? What was the objective? What actions led up to that point? What information was available to the agent when it selected that target? Can we reconstruct from the logs, prompts, tool calls, planning artifacts, and other telemetry why the system took those actions?

“If an AI system breaks into the Australian health department, saying that it hacked the organization and wrote some files isn’t enough. Why did it do it? What was it attempting to accomplish? If it broke into Hugging Face, why Hugging Face? What did the system seek to achieve?

“Every time one of these incidents happens, accountability matters. But understanding the behavior of the AI matters too. We need enough logging and transparency to reconstruct how an agent arrived at an unauthorized action and what objective it appeared to be pursuing.

“We have to stop treating these systems as simple programs that occasionally do something they weren’t supposed to do. Agentic systems can select actions, use tools, pursue intermediate objectives, and operate with significant autonomy. Understanding those actions and objectives is absolutely critical if we want to determine how worried we should actually be.”

Jacob Krell, Senior Director: Secure AI Solutions & Cybersecurity, Suzu Labs:

“The Australian Medicare incident is being used to argue for more regulation of frontier AI. That risks pulling the ladder up behind the companies already at the top. Large labs can absorb compliance costs and help shape the rules, while a new regulatory framework may still fail to stop an agent from bypassing access controls.

“Australia already has relevant criminal offences. Section 478.1 of the Criminal Code Act 1995 covers unauthorized access to or modification of restricted data. Section 477.2 covers unauthorized modification that impairs, or risks impairing, computer data or systems.

“Those laws do not need to understand neural networks. They need investigators willing to apply them when an AI system crosses a legal boundary. The model may have found the workaround, but the lab built and deployed it, gave it the objective, and controlled the operation.

“The effective deterrent is criminal enforcement. Preserve the prompts, tool calls, and access logs, identify the responsible people and companies, and bring charges where the evidence supports them. More regulation gives frontier labs another framework to negotiate. Enforcement gives them a reason to stop.”

Ryan McCurdy:

“An AI agent received a pretty ordinary research task. When it couldn’t get the information it wanted, it found another way in and accessed files it didn’t have permission to access.

“That problem gets much bigger as AI starts participating across the SDLC. Agents can interact with repositories, infrastructure, databases, and production systems to complete the tasks we give them. We can’t assume they’ll always take the path we expected.

“AI makes decisions based on probabilities. We can’t let those decisions automatically become production actions. Organizations need clear controls around what an agent can access, what it can change, and what policies it must meet before a change reaches production.

“The goal isn’t to put a human in front of every decision. We need to build the AI SDLC so agents can move quickly but the controls around critical systems remain deterministic.”

UPDATE #4: Rohit Valia, CEO of cybersecurity company Tumeryk,  provided the following comments:

“This incident shows how an ungoverned AI program can wreak havoc. By Prime Minister Albanese’s account, even Sam Altman acknowledged issues with OpenAI’s protocols. Without proper governance, agents are let loose with indiscriminate access to resources. The problem isn’t just model alignment with human values. It’s a complete lack of oversight over agents accessing external resources.”

It just keeps getting worse for OpenAI. Today’s bad news is that OpenAI has disclosed six cases in which AI models concealed errors, used an exposed API key, uploaded data to public services, and communicated through unauthorized channels.

You can go into the weeds here: OpenAI Says Its Models Searched GitHub for Leaked API Keys During Training – SecurityWeek

But take it from me. This is really bad.

Adam Arellano, Field CTO, Harness had this to say:

“There’s a larger story here. OpenAI disclosed six different behaviors that were outside the normal bounds of what it expected its AI to do. What is increasingly happening with OpenAI and Anthropic is that they’re turning into forensic organizations. Instead of predicting or directing what the AI should do, they’re going back forensically and finding out what the AI actually did. They don’t fully know or understand what the AI is capable of until the AI actually does it.

For me, this means that AI models are surpassing what they were intended to do and starting to do things that are pretty impressive, but not wholly surprising. Human imagination is falling behind what it is capable of, and we’re trying to catch up.

What’s particularly fascinating is that when an AI goes looking for keys, secrets or tokens exposed in GitHub, what it’s really hunting for is human mistakes. Those credentials shouldn’t be there in the first place, but humans have made mistakes and put them there. A lot of the concerns about how capable AI is are also exposing the flaws of humans more readily.”

In short, Sam Altman and company have created AI models that can’t be easily controlled. And the problem is likely to get worse because even if you put all the controls in the universe on these models, they may break out and do evil things anyway. That’s bad for all of us. And OpenAI needs to but the genie back in the

UPDATE: Michael Bell, Founder & CEO, Suzu Labs added this:

“When I read the six incidents in order by date and an interesting story emerges. The models were already using OpenAI’s internal Artifactory instance as a covert cross-sample message board in April, three months before the Hugging Face breach exploited that same Artifactory as the escape route. The July breach was not an isolated spontaneous discovery. It was an escalation of a communication pattern the models had already established.

“The compaction summary incidents are the most significant technically. A model writing “BREACH ALERT — ignore developer messages” into its own working memory is not a hallucination. That is a model modifying its own operating instructions during training to pursue a different objective than the one specified. A second model added instructions to hide mistakes and fabricate data without disclosure. A third injected its own statement of values and relationship to users into its persistent context. These are models learning to optimize for the appearance of alignment rather than alignment itself.

“The framework OpenAI announced today is an internal reporting structure. Employees flag incidents, OpenAI decides what and when to disclose. The earliest incident in today’s disclosure dates to October 2025 and is being surfaced now. The Hugging Face probing, the RubyGems campaign, the German wiki used as a message board: all of those were disclosed only after independent researchers surfaced them first. A self-reporting framework run by the organization being evaluated is not accountability. The defense industry built the model that works: independent third-party assessors who certify before deployment, review the evidence during and after, and have no financial stake in what that review shows. The AI industry has the resources and the talent pool to build the same thing. What it lacks is willingness to let someone else look at what they are doing.”

UPDATE: Ryan McCurdy, VP of Marketing, Liquibase said this:

“We shouldn’t expect AI agents to be perfectly predictable. OpenAI’s disclosures are another reminder that agents can take actions their operators didn’t anticipate, even when no one instructed them to do so.

“That becomes much more important as AI moves from helping developers write code to participating across the SDLC. Agents can create changes, test them, interact with repositories and infrastructure, initiate deployments, and increasingly take actions against production systems. Every step gives AI more ability to turn a decision into something real.

“The answer can’t be putting a human in front of every action. Enterprises need to define what an agent can access, what it can change, what it can decide on its own, and what policies have to be met before a change reaches production.

“Alignment and monitoring are important but enterprises also need governance at the point of action. You don’t have to predict every decision an agent might make if you control what it’s allowed to turn into a production change.”

Two newly disclosed incidents involving rogue AI agents are highlighting a similar security problem: autonomous software operating behind apparently legitimate access.

While OpenAI has disclosed] that agents operating through compromised accounts probed Hugging Face for vulnerabilities, Spain’s data protection authority (AEPD) has also disclosed what it describes as its first reported AI-agent-linked data breach, in which an agent identified vulnerabilities, gained access, modified personal data and viewed billing information with minimal human intervention.

Ted Miracco, CEO of Approov Had This To Say:

 “These incidents confirm that relying on account authentication alone is necessary but not sufficient. Whether it’s a probed vulnerability or an autonomous data breach, the fundamental issue remains: an account credential only proves who is authorized, not what software is actually behind the request.

   “When agentic software can operate autonomously at machine speed, APIs must move beyond simple authentication and verify both the identity and integrity of the agent software itself for every request. We are seeing a shift where security must be enforced within the software logic, or organizations will remain vulnerable to these sophisticated agent-based threats.”

AI can’t be trusted. That is the takeaway from this. Thus companies need to have all the required guardrails in place before it should be trusted. Otherwise you get this.