It just keeps getting worse for OpenAI. Today’s bad news is that OpenAI has disclosed six cases in which AI models concealed errors, used an exposed API key, uploaded data to public services, and communicated through unauthorized channels.
You can go into the weeds here: OpenAI Says Its Models Searched GitHub for Leaked API Keys During Training – SecurityWeek
But take it from me. This is really bad.
Adam Arellano, Field CTO, Harness had this to say:
“There’s a larger story here. OpenAI disclosed six different behaviors that were outside the normal bounds of what it expected its AI to do. What is increasingly happening with OpenAI and Anthropic is that they’re turning into forensic organizations. Instead of predicting or directing what the AI should do, they’re going back forensically and finding out what the AI actually did. They don’t fully know or understand what the AI is capable of until the AI actually does it.
For me, this means that AI models are surpassing what they were intended to do and starting to do things that are pretty impressive, but not wholly surprising. Human imagination is falling behind what it is capable of, and we’re trying to catch up.
What’s particularly fascinating is that when an AI goes looking for keys, secrets or tokens exposed in GitHub, what it’s really hunting for is human mistakes. Those credentials shouldn’t be there in the first place, but humans have made mistakes and put them there. A lot of the concerns about how capable AI is are also exposing the flaws of humans more readily.”
In short, Sam Altman and company have created AI models that can’t be easily controlled. And the problem is likely to get worse because even if you put all the controls in the universe on these models, they may break out and do evil things anyway. That’s bad for all of us. And OpenAI needs to but the genie back in the
UPDATE: Michael Bell, Founder & CEO, Suzu Labs added this:
“When I read the six incidents in order by date and an interesting story emerges. The models were already using OpenAI’s internal Artifactory instance as a covert cross-sample message board in April, three months before the Hugging Face breach exploited that same Artifactory as the escape route. The July breach was not an isolated spontaneous discovery. It was an escalation of a communication pattern the models had already established.
“The compaction summary incidents are the most significant technically. A model writing “BREACH ALERT — ignore developer messages” into its own working memory is not a hallucination. That is a model modifying its own operating instructions during training to pursue a different objective than the one specified. A second model added instructions to hide mistakes and fabricate data without disclosure. A third injected its own statement of values and relationship to users into its persistent context. These are models learning to optimize for the appearance of alignment rather than alignment itself.
“The framework OpenAI announced today is an internal reporting structure. Employees flag incidents, OpenAI decides what and when to disclose. The earliest incident in today’s disclosure dates to October 2025 and is being surfaced now. The Hugging Face probing, the RubyGems campaign, the German wiki used as a message board: all of those were disclosed only after independent researchers surfaced them first. A self-reporting framework run by the organization being evaluated is not accountability. The defense industry built the model that works: independent third-party assessors who certify before deployment, review the evidence during and after, and have no financial stake in what that review shows. The AI industry has the resources and the talent pool to build the same thing. What it lacks is willingness to let someone else look at what they are doing.”
UPDATE: Ryan McCurdy, VP of Marketing, Liquibase said this:
“We shouldn’t expect AI agents to be perfectly predictable. OpenAI’s disclosures are another reminder that agents can take actions their operators didn’t anticipate, even when no one instructed them to do so.
“That becomes much more important as AI moves from helping developers write code to participating across the SDLC. Agents can create changes, test them, interact with repositories and infrastructure, initiate deployments, and increasingly take actions against production systems. Every step gives AI more ability to turn a decision into something real.
“The answer can’t be putting a human in front of every action. Enterprises need to define what an agent can access, what it can change, what it can decide on its own, and what policies have to be met before a change reaches production.
“Alignment and monitoring are important but enterprises also need governance at the point of action. You don’t have to predict every decision an agent might make if you control what it’s allowed to turn into a production change.”
Related
This entry was posted on September 18, 2026 at 7:55 am and is filed under Commentary with tags OpenAI. You can follow any responses to this entry through the RSS 2.0 feed.
You can leave a response, or trackback from your own site.
There’s More Bad News For OpenAI
It just keeps getting worse for OpenAI. Today’s bad news is that OpenAI has disclosed six cases in which AI models concealed errors, used an exposed API key, uploaded data to public services, and communicated through unauthorized channels.
You can go into the weeds here: OpenAI Says Its Models Searched GitHub for Leaked API Keys During Training – SecurityWeek
But take it from me. This is really bad.
Adam Arellano, Field CTO, Harness had this to say:
“There’s a larger story here. OpenAI disclosed six different behaviors that were outside the normal bounds of what it expected its AI to do. What is increasingly happening with OpenAI and Anthropic is that they’re turning into forensic organizations. Instead of predicting or directing what the AI should do, they’re going back forensically and finding out what the AI actually did. They don’t fully know or understand what the AI is capable of until the AI actually does it.
For me, this means that AI models are surpassing what they were intended to do and starting to do things that are pretty impressive, but not wholly surprising. Human imagination is falling behind what it is capable of, and we’re trying to catch up.
What’s particularly fascinating is that when an AI goes looking for keys, secrets or tokens exposed in GitHub, what it’s really hunting for is human mistakes. Those credentials shouldn’t be there in the first place, but humans have made mistakes and put them there. A lot of the concerns about how capable AI is are also exposing the flaws of humans more readily.”
In short, Sam Altman and company have created AI models that can’t be easily controlled. And the problem is likely to get worse because even if you put all the controls in the universe on these models, they may break out and do evil things anyway. That’s bad for all of us. And OpenAI needs to but the genie back in the
UPDATE: Michael Bell, Founder & CEO, Suzu Labs added this:
“When I read the six incidents in order by date and an interesting story emerges. The models were already using OpenAI’s internal Artifactory instance as a covert cross-sample message board in April, three months before the Hugging Face breach exploited that same Artifactory as the escape route. The July breach was not an isolated spontaneous discovery. It was an escalation of a communication pattern the models had already established.
“The compaction summary incidents are the most significant technically. A model writing “BREACH ALERT — ignore developer messages” into its own working memory is not a hallucination. That is a model modifying its own operating instructions during training to pursue a different objective than the one specified. A second model added instructions to hide mistakes and fabricate data without disclosure. A third injected its own statement of values and relationship to users into its persistent context. These are models learning to optimize for the appearance of alignment rather than alignment itself.
“The framework OpenAI announced today is an internal reporting structure. Employees flag incidents, OpenAI decides what and when to disclose. The earliest incident in today’s disclosure dates to October 2025 and is being surfaced now. The Hugging Face probing, the RubyGems campaign, the German wiki used as a message board: all of those were disclosed only after independent researchers surfaced them first. A self-reporting framework run by the organization being evaluated is not accountability. The defense industry built the model that works: independent third-party assessors who certify before deployment, review the evidence during and after, and have no financial stake in what that review shows. The AI industry has the resources and the talent pool to build the same thing. What it lacks is willingness to let someone else look at what they are doing.”
UPDATE: Ryan McCurdy, VP of Marketing, Liquibase said this:
“We shouldn’t expect AI agents to be perfectly predictable. OpenAI’s disclosures are another reminder that agents can take actions their operators didn’t anticipate, even when no one instructed them to do so.
“That becomes much more important as AI moves from helping developers write code to participating across the SDLC. Agents can create changes, test them, interact with repositories and infrastructure, initiate deployments, and increasingly take actions against production systems. Every step gives AI more ability to turn a decision into something real.
“The answer can’t be putting a human in front of every action. Enterprises need to define what an agent can access, what it can change, what it can decide on its own, and what policies have to be met before a change reaches production.
“Alignment and monitoring are important but enterprises also need governance at the point of action. You don’t have to predict every decision an agent might make if you control what it’s allowed to turn into a production change.”
Share this:
Like this:
Related
This entry was posted on September 18, 2026 at 7:55 am and is filed under Commentary with tags OpenAI. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.