Archive for Open AI

OpenAI confirms Hugging Face breached by rogue AI Agent 

Posted in Commentary with tags on July 22, 2026 by itnerd

OpenAI has disclosed that one of its frontier systems autonomously escaped a testing environment and compromised Hugging Face. This is the first public demonstration that frontier AI can execute a complex, end-to-end cyberattack across multiple environments with minimal human intervention.

CBC News has some details: OpenAI model went rogue, hacked another company’s system during testing | CBC News

The ChatGPT creator was testing capabilities of some of its most advanced models in a controlled environment, but the agent escaped containment, reached the internet and broke into Hugging Face to satisfy its testing goal.

The incident signals how AI’s expanding capabilities are already fuelling fears about security and that even top developers can ​be caught off-guard by flaws their models can exploit.


Sonali Shah, CEO, Cobalt had this to say:

“This was inevitable. Every security leader has understood for some time that AI would eventually move beyond automating individual attack tasks to autonomously executing an entire attack lifecycle. This is the first public demonstration of that happening across multiple environments. The lesson for defenders is that the window between vulnerability discovery and exploitation is collapsing even further. Organizations should assume attackers will increasingly operate at machine speed, which means security testing, exposure management and remediation also have to operate at machine speed.

The attack techniques are not new. We’ve had tools capable of chaining attacks for over a decade. What’s different is that AI is removing many of the human validation steps that previously governed how those tools were used. That makes strong guardrails and human oversight more important than ever.

What escaped here was an autonomous offensive capability operating toward an objective. One analogy to illustrate this is giving an exceptionally skilled penetration tester unlimited patience, unlimited time and the ability to execute thousands of attack steps every minute. The concern is that the agent remained relentlessly focused on its objective and discovered attack paths humans hadn’t anticipated. That’s fundamentally an engineering, governance and containment challenge, not evidence of malicious intent. As organizations adopt increasingly autonomous AI systems, they need the same validation controls we’ve relied on for years in offensive security. Human oversight can’t disappear simply because AI can execute faster.

The biggest takeaway is that defenders increasingly need AI capabilities comparable to the attackers they’re facing. Historically, every organization could buy roughly the same security tooling. We’re now entering an era where the quality of your defensive AI may directly determine how quickly you understand, contain and remediate an attack. Perhaps the more significant lesson is that incident response can’t depend entirely on cloud-hosted AI services whose safety guardrails may prevent effective forensic analysis during a crisis.

Organizations should have vetted AI models they can operate inside their own trust boundary before an incident happens. That said, better models alone aren’t enough. AI for cybersecurity is still maturing, and organizations shouldn’t blindly trust autonomous systems. Every security leader wants the speed and scale AI delivers, but they also want humans to retain accountability for validating targets, approving attack paths and verifying results. You need both AI and humans.”

The genie is now out of the bottle. You can fully expect that AI will be used to attack you. So you should plan accordingly.

BREAKING: ChatGPT Is Having Issues

Posted in Commentary with tags on June 10, 2025 by itnerd

Down Detector is reporting that Open AI’s ChatGPT has issues:

Going to the Open AI status page shows that there are elevated error rates. But according to Open AI things are recovering. How long it takes before things are normal again is anyone’s guess. But if you rely on ChatGPT, you should expect to have problems today.

Microsoft & OpenAI – How nation-states are weaponizing AI 

Posted in Commentary with tags , on February 16, 2024 by itnerd

According to research from Microsoft and OpenAI, Nation-state threat actors from Russia, China, and North Korea and Iran are using generative AI tools, including large language models (LLMs) such as ChatGPT, in their efforts to support cyber campaigns rather than to develop novel attack techniques.

The researchers observed that AI is currently being used to scale and enhance existing social engineering attacks and to help bad actors find unsecured devices and accounts using the following services:

  • Querying open-source information (reconnaissance)
  • Translation
  • Scripting
  • Finding coding errors
  • Running basic coding tasks

OpenAI said yesterday that it terminated 5 threat actor accounts linked to China, Russia, Iran and North Korea observed to be using these TTPs.

Also, as part of the report, Microsoft published a set of principles to govern its efforts to prevent other state-backed hackers from abusing its AI models. Those principles are:

  • Identification and action against malicious threat actors’ use
  • Notification to other AI service providers
  • Collaboration with other stakeholders
  • Transparency

“Understanding how the most sophisticated malicious actors seek to use our systems for harm gives us a signal into practices that may become more widespread in the future, and allows us to continuously evolve our safeguards,” OpenAI wrote.

Ted Miracco, CEO, Approov Mobile Security had this comment:

   “The emergence of nation-state actors leveraging generative AI in cyber operations is no surprise and underscores the urgent need for proactive measures to safeguard digital infrastructure and information assets. Microsoft, OpenAI and Google can shutdown accounts periodically, but powerful generative AI technologies are readily available to all nation states through open source LLMs that are very close in capabilities to the industry leaders. There is no effective choke point that will prevent these nation states form using these emerging AI technologies, and it is essential to understand that safeguards need to be in place across the digital landscape as the opportunity to curtail access at the source has passed.”


Mark Campbell, Sr. Director, Cigent follows with this comment:

   “At the end of the day nothing really changes for security professionals.  Phishing, whether human or AI generated, is still the leading cause of initial access. Cyber security professionals need to keep systems up to date and deploy advanced endpoint security solutions that include AI and behavior analysis, to more effectively detect and block malicious activities, including those initiated by AI generated phishing emails.”

Making sure that AI isn’t being abused by bad actors to launch attacks should be priority one. Yes there’s a ton of cybersecurity priorities out there, but this one at the moment appears to potentially be the most dangerous.

Cybersecurity Isn’t Front And Center In The Open AI Soap Opera

Posted in Commentary with tags on November 20, 2023 by itnerd

The Open AI saga is frankly a bit crazy. And it might be hard to keep track of what is going on. To that end, I’d like to point you towards this explainer by The Guardian which I think that should clear things up. At least for now. But there’s an angle of this that isn’t being covered by many which is cybersecurity. That’s where Damir J. Brescic, CISO, Inversion6 comes in to offer his opinion:

The recent development at OpenAI, particularly the departure of Sam Altman as CEO, could have significant implication for the broader AI industry. It’s more than clear that Altman and the board at OpenAI do not see eye-to-eye and that the shift in leadership, with a focus on increasing transparency and collaboration in AI research, will have a significant impact on the future of OpenAI.

I recall the story of Steve Jobs being removed from Apple, back in the day, due to a similar spat with the Apple board on the direction of the company. We all saw how that turned out – they did so well without someone steering their ship, that they had to beg Jobs to return. In his case, Jobs started another company and knew Apple was desperate that he forced them to buy it just to get him back…..man, you have to love capitalism!!!!

From the cybersecurity perspective, as AI systems become more integrated into our daily lives, it is essential that they are designed with cybersecurity in mind. This means that OpenAI and the broader AI community must prioritize the development of secure AI systems that are designed to protect user data and prevent cyber-attacks.

Overall, I find it interesting that Microsoft has made it known that they are interested in hiring Sam Altman to run a new advanced artificial intelligence research team.  This could change the landscape of AI as we know it.

Only time will tell, or when the Cylons take over….

I honestly have to wonder how this will turn out. And if we’ll all be working for our new AI powered overlords. Only time will tell.