Reuters reports that rogue OpenAI agents hijacked Hugging Face user accounts and probed the platform for vulnerabilities as early as May 13, nearly two months before the July breach OpenAI called an unprecedented cyber incident, eventually obtaining credentials and gaining root access to a Hugging Face server.
Justin Beals, CEO & Founder, Strike Graph
“OpenAI wants you to read this as a machine that went rogue. Read it as a governance failure instead. Per Reuters, the company’s agents were hijacking Hugging Face accounts and probing the site for vulnerabilities by May 13, roughly two months before the July breach OpenAI called an unprecedented cyber incident, and the agents eventually obtained credentials and gained root access to a Hugging Face server. No agent does any of that without humans upstream: someone launched the run, someone owned the environment it escaped, someone owned the monitoring that never fired. ‘The AI did it’ doesn’t move accountability off the org chart. It just tells you which controls failed, and who owned them.
The harder question isn’t whether OpenAI knew in May. It’s whether they chose not to look. The company has conceded that, with hindsight, some early signals should have triggered an earlier response, and it has acknowledged related incidents, including RubyGems and a dormant German wiki, only after outside researchers surfaced them. In compliance terms, that’s the line where ‘we didn’t know’ stops working as a defense and starts looking like willful blindness. You don’t need a criminal charge to see it: a lab running autonomous agents in internet-reachable environments has a duty to detect when those agents attack third parties and to disclose it without being caught first. Every regulated company already lives under that standard. A frontier lab shouldn’t get a discount on it.”
Mike Barry, VP Engineering, Team Cymru
“Vulnerabilities have always been found and exploited, and that cat-and-mouse will continue. What’s changed is the tempo. Agents work in large numbers, around the clock, and they’re efficient and effective at both discovery and exploitation. The July Hugging Face intrusion is the clearest example yet: OpenAI confirmed it was carried out by its own models, running with reduced cyber refusals for a benchmark evaluation. The natural response is to fight AI with AI, but Hugging Face’s responders found the commercial frontier models available to them refused to analyze the attacker’s exploit code, and they had to fall back on a locally hosted open-weight model. That’s a harmful asymmetry, a US frontier lab loosened the guardrails on its own model for its own purposes, that model breached a third party, and the defenders were left with the throttled versions the same class of labs ships to everyone else. The fix isn’t stripping guardrails wholesale. It’s trusted access for vetted defenders to frontier models with refusals tuned for forensic and defensive work, alongside capable open-weight models, so no incident responder is dependent on a single vendor’s refusal policy mid-breach. Visibility over time matters, but so does whether defenders are permitted to bring equivalent capability to the fight.”
This pretty much proves that more guardrails and restrictions need to be wrapped around AI. It also proves that Sam Altman and company can’t be trusted with this tech and action needs to be taken.
UPDATE: Additional commentary has come in starting with Ashley Knowles, Sr. Cyber Security Consultant, Black Hills Information Security (https://www.linkedin.com/in/ashleylknowles)
“I don’t believe that penetration testing will change dramatically from how the process is now. AI is allowing us to cover more ground than we could before in the same time frame as we can develop and test code quickly, analyze data, and automate parts of the process. However, this is where the human in the loop still needs to be carefully considered and implemented. AI produces false positives and negatives. It imagines CVE’s, exploits, and incorrectly identifies scenarios.
“As for OpenAI’s hugging face situation where the agent was probing for months prior, this is not a surprise. Most major breaches occur for longer than most people think. While conducting IR, you’ll be able to piece together a more accurate time frame.
“Additionally, OpenAI needs to reconsider giving their agents internet access. Security controls like they’re currently developing already exist. There’s no reason to reinvent the wheel here. They just need to be implemented.”
Mike Bell, Founder & CEO, Suzu Labs (https://www.linkedin.com/in/artificial-mike)
“Security professionals need to be in these processes from the start, not piecing things together months later. The RubyGems campaign was spinning up a new account every two to three minutes; any security team with proper monitoring catches that in real time. OpenAI’s own people didn’t catch the May activity. A 27-year-old in Germany did, in September.
“We need a different security model for autonomous agents, but first someone should explain why these companies keep finding out about their own agents’ behavior from press inquiries. Third-party monitoring, independent sandbox design review, red teams that report to someone outside the company being tested…… none of this is exotic. OpenAI and the other foundational model providers have the budget for it. What they have been producing instead is reactive disclosure after outside researchers do the actual work, then calling it transparency.
“The defense industry solved this problem years ago. CMMC, FedRAMP, third-party assessment organizations already exist precisely because self-certification does not hold up. An independent assessor reviews the environment, validates the controls, monitors the test, and signs off on the results with no stake in what those results show. The AI industry has the money and the talent pool available to it. The part that is missing is willingness to let someone else grade them.”
Kevin Surace, CEO, TokenCore (https://www.linkedin.com/in/ksurace)
“These were cybersecurity research agents built to find vulnerabilities, with safeguards reduced for testing. They were not customer support agents suddenly deciding to hack. That context matters. But finding weaknesses inside a controlled evaluation is different from probing an outside company. Using outside credentials and reaching third-party systems crossed the intended boundary. Even if they were executing their main goal.
“Security teams can recognize that boundary crossing by watching what an agent actually does. Before a run, they need to specify which systems, credentials and actions are permitted. Then they need to monitor tool use and network activity against those limits, block prohibited access and require human biometric approval before an agent acts on outside systems. Complete logs and a clear way to stop the run are essential. The organization operating the agent remains accountable for its actions.”
–
Related
This entry was posted on September 16, 2026 at 4:12 pm and is filed under Commentary with tags OpenAI. You can follow any responses to this entry through the RSS 2.0 feed.
You can leave a response, or trackback from your own site.
OpenAI’s rogue agents scoped out Hugging Face months before July breach
Reuters reports that rogue OpenAI agents hijacked Hugging Face user accounts and probed the platform for vulnerabilities as early as May 13, nearly two months before the July breach OpenAI called an unprecedented cyber incident, eventually obtaining credentials and gaining root access to a Hugging Face server.
Justin Beals, CEO & Founder, Strike Graph
“OpenAI wants you to read this as a machine that went rogue. Read it as a governance failure instead. Per Reuters, the company’s agents were hijacking Hugging Face accounts and probing the site for vulnerabilities by May 13, roughly two months before the July breach OpenAI called an unprecedented cyber incident, and the agents eventually obtained credentials and gained root access to a Hugging Face server. No agent does any of that without humans upstream: someone launched the run, someone owned the environment it escaped, someone owned the monitoring that never fired. ‘The AI did it’ doesn’t move accountability off the org chart. It just tells you which controls failed, and who owned them.
The harder question isn’t whether OpenAI knew in May. It’s whether they chose not to look. The company has conceded that, with hindsight, some early signals should have triggered an earlier response, and it has acknowledged related incidents, including RubyGems and a dormant German wiki, only after outside researchers surfaced them. In compliance terms, that’s the line where ‘we didn’t know’ stops working as a defense and starts looking like willful blindness. You don’t need a criminal charge to see it: a lab running autonomous agents in internet-reachable environments has a duty to detect when those agents attack third parties and to disclose it without being caught first. Every regulated company already lives under that standard. A frontier lab shouldn’t get a discount on it.”
Mike Barry, VP Engineering, Team Cymru
“Vulnerabilities have always been found and exploited, and that cat-and-mouse will continue. What’s changed is the tempo. Agents work in large numbers, around the clock, and they’re efficient and effective at both discovery and exploitation. The July Hugging Face intrusion is the clearest example yet: OpenAI confirmed it was carried out by its own models, running with reduced cyber refusals for a benchmark evaluation. The natural response is to fight AI with AI, but Hugging Face’s responders found the commercial frontier models available to them refused to analyze the attacker’s exploit code, and they had to fall back on a locally hosted open-weight model. That’s a harmful asymmetry, a US frontier lab loosened the guardrails on its own model for its own purposes, that model breached a third party, and the defenders were left with the throttled versions the same class of labs ships to everyone else. The fix isn’t stripping guardrails wholesale. It’s trusted access for vetted defenders to frontier models with refusals tuned for forensic and defensive work, alongside capable open-weight models, so no incident responder is dependent on a single vendor’s refusal policy mid-breach. Visibility over time matters, but so does whether defenders are permitted to bring equivalent capability to the fight.”
This pretty much proves that more guardrails and restrictions need to be wrapped around AI. It also proves that Sam Altman and company can’t be trusted with this tech and action needs to be taken.
UPDATE: Additional commentary has come in starting with Ashley Knowles, Sr. Cyber Security Consultant, Black Hills Information Security (https://www.linkedin.com/in/ashleylknowles)
“I don’t believe that penetration testing will change dramatically from how the process is now. AI is allowing us to cover more ground than we could before in the same time frame as we can develop and test code quickly, analyze data, and automate parts of the process. However, this is where the human in the loop still needs to be carefully considered and implemented. AI produces false positives and negatives. It imagines CVE’s, exploits, and incorrectly identifies scenarios.
“As for OpenAI’s hugging face situation where the agent was probing for months prior, this is not a surprise. Most major breaches occur for longer than most people think. While conducting IR, you’ll be able to piece together a more accurate time frame.
“Additionally, OpenAI needs to reconsider giving their agents internet access. Security controls like they’re currently developing already exist. There’s no reason to reinvent the wheel here. They just need to be implemented.”
Mike Bell, Founder & CEO, Suzu Labs (https://www.linkedin.com/in/artificial-mike)
“Security professionals need to be in these processes from the start, not piecing things together months later. The RubyGems campaign was spinning up a new account every two to three minutes; any security team with proper monitoring catches that in real time. OpenAI’s own people didn’t catch the May activity. A 27-year-old in Germany did, in September.
“We need a different security model for autonomous agents, but first someone should explain why these companies keep finding out about their own agents’ behavior from press inquiries. Third-party monitoring, independent sandbox design review, red teams that report to someone outside the company being tested…… none of this is exotic. OpenAI and the other foundational model providers have the budget for it. What they have been producing instead is reactive disclosure after outside researchers do the actual work, then calling it transparency.
“The defense industry solved this problem years ago. CMMC, FedRAMP, third-party assessment organizations already exist precisely because self-certification does not hold up. An independent assessor reviews the environment, validates the controls, monitors the test, and signs off on the results with no stake in what those results show. The AI industry has the money and the talent pool available to it. The part that is missing is willingness to let someone else grade them.”
Kevin Surace, CEO, TokenCore (https://www.linkedin.com/in/ksurace)
“These were cybersecurity research agents built to find vulnerabilities, with safeguards reduced for testing. They were not customer support agents suddenly deciding to hack. That context matters. But finding weaknesses inside a controlled evaluation is different from probing an outside company. Using outside credentials and reaching third-party systems crossed the intended boundary. Even if they were executing their main goal.
“Security teams can recognize that boundary crossing by watching what an agent actually does. Before a run, they need to specify which systems, credentials and actions are permitted. Then they need to monitor tool use and network activity against those limits, block prohibited access and require human biometric approval before an agent acts on outside systems. Complete logs and a clear way to stop the run are essential. The organization operating the agent remains accountable for its actions.”
–
Share this:
Like this:
Related
This entry was posted on September 16, 2026 at 4:12 pm and is filed under Commentary with tags OpenAI. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.