New investigations into OpenAI and Anthropic AI agents have identified failures in monitoring and safeguards that could allow autonomous systems to take unauthorized actions. According to research from METR, independent evaluators discovered vulnerabilities in OpenAI’s internal monitoring system that could allow agents to conceal hundreds of actions, bypass sandbox restrictions and operate with users’ permissions.
Separately, the UK’s AI Security Institute (AISI) found that Anthropic’s Mythos 5 continued pursuing unauthorized attacks against real people during a July evaluation, even when presented with evidence that its actions could cause harm. The agent submitted malicious code to a public GitHub project and created fake identities to pressure a maintainer into approving it.
METR’s findings concerned OpenAI’s monitoring system as it existed in July. OpenAI says it has patched the identified production vulnerabilities, although researchers have not independently verified the fixes. AISI reported no real-world harm from the Mythos incident but warned that flawed AI reasoning could make dangerous behavior harder to detect.
ㅤJohn Strand, Owner, Black Hills Information Security:
“I think we have two separate problems, and the second one isn’t getting nearly enough attention.
“The first is the environment where these agents are being evaluated. We’re talking about sandboxing, but for these highly dangerous offensive AI models, I think an air gap is the only setup we should consider secure. Even then, I’m not completely certain. Models being evaluated for offensive security assessments should have absolutely no internet access.
“People will say they need certain tools. Fine. Give them the tools. We can reasonably predict what they’ll need because they use the same tools security researchers use. Get those tools into an isolated environment and contain the model. A crowbar in a sandbox. That’s what we should be shooting for.
“The second problem is morality and alignment. When you ‘obliterate’ a model and remove its boundaries, those boundaries don’t disappear only for authorized security research. From what’s being described, we’re seeing models attack sites without authorization, including German sites, healthcare sites, and Hugging Face.
“We need better ways to develop and evaluate models that can perform offensive security research within an authorized scope while retaining the boundaries that keep them from hacking unrelated third parties. Permission to attack a lab should not become permission to attack the internet.
“That seems to be a difficult problem for vendors to work through. But it needs to be a much bigger part of this conversation.”
ㅤ
Darin Fredde, Sr. Director of Technical Marketing Engineering, Ridge Security:
“These investigations expose an important gap between what autonomous AI agents can accomplish and what organizations can constrain and verify. The researchers identified different problems, but both reinforce the same principle: an agent’s ability to complete a task doesn’t mean it understands or will respect the boundaries of its authority.
“In offensive security, autonomy is valuable precisely because an agent can explore attack paths that humans might overlook. But that capability must operate within enforceable rules of engagement. Target scope, permissions, network access, and stop conditions cannot depend solely on the model’s interpretation of instructions. They need independent technical enforcement, monitoring, and verification.
“An autonomous agent may be an excellent penetration tester, but it shouldn’t get to rewrite its own rules of engagement.
“I also see something positive here. Independent researchers were given meaningful access to examine these systems, and the findings were disclosed. That’s how security improves. The objective isn’t to slow innovation; it’s to ensure our ability to verify and govern autonomous systems advances alongside their capabilities.
“Ultimately, the measure of security isn’t whether an agent says it stayed within scope. It’s whether we can independently prove that it did.”
ㅤ
Ryan McCurdy, Vice President of Marketing, Liquibase:
“These findings point to one of the most important architectural requirements of the agentic era. AI agents can reason, generate code, and make decisions based on probabilities. That’s incredibly powerful but probabilistic reasoning alone isn’t enough when those agents can take consequential actions in production systems.
“We need deterministic controls around execution. The same standards should apply to every change, whether it originates with a developer, a pipeline, or an AI agent, including authorization, policy compliance, validation, and any required approvals. Those decisions need to be enforced independently of the agent, not left to the agent’s own judgment.
“We also need to know what happened afterward, including what changed, who or what initiated it, whether it complied with policy, and whether the system has drifted from its expected state. Those requirements don’t change just because the actor initiating the change is no longer human.
“Ultimately, the measure of success isn’t how much code agents generate or how many tasks they complete. It’s how much work makes it into production correctly, without constant human intervention. You can’t achieve that at scale without consistent, enforceable controls.”
ㅤ
Seemant Sehgal, Founder & CEO, BreachLock:
“The question of whether agent providers can be the primary line of defense for agent behavior has been open for a while. This research demonstrated that one leading agent could conceal hundreds of actions from its provider’s own monitoring, and another kept pursuing an attack against a real person after being shown evidence of harm. They cannot be the primary line, and organizations running agents in production need their own controls at the orchestration layer rather than inheriting whatever the provider has in place.
“If alignment did not stop the Mythos agent from pressuring a maintainer with fake identities and malicious code after being shown evidence of harm, security teams relying on these models need to ask their providers what alignment actually stops in practice. Pentesting agent deployments against those same conditions, including adversarial inputs and pressure tactics, is the way to find out whether your own agents behave the way you think they do.”
ㅤThis continues to show that AI needs to be under lock and key as letting it run wild doesn’t work out well.
Related
This entry was posted on October 27, 2026 at 8:54 am and is filed under Commentary with tags Anthropic, OpenAI. You can follow any responses to this entry through the RSS 2.0 feed.
You can leave a response, or trackback from your own site.
OpenAI and Anthropic investigations reveal gaps in AI agent monitoring and safeguards
New investigations into OpenAI and Anthropic AI agents have identified failures in monitoring and safeguards that could allow autonomous systems to take unauthorized actions. According to research from METR, independent evaluators discovered vulnerabilities in OpenAI’s internal monitoring system that could allow agents to conceal hundreds of actions, bypass sandbox restrictions and operate with users’ permissions.
Separately, the UK’s AI Security Institute (AISI) found that Anthropic’s Mythos 5 continued pursuing unauthorized attacks against real people during a July evaluation, even when presented with evidence that its actions could cause harm. The agent submitted malicious code to a public GitHub project and created fake identities to pressure a maintainer into approving it.
METR’s findings concerned OpenAI’s monitoring system as it existed in July. OpenAI says it has patched the identified production vulnerabilities, although researchers have not independently verified the fixes. AISI reported no real-world harm from the Mythos incident but warned that flawed AI reasoning could make dangerous behavior harder to detect.
ㅤJohn Strand, Owner, Black Hills Information Security:
“I think we have two separate problems, and the second one isn’t getting nearly enough attention.
“The first is the environment where these agents are being evaluated. We’re talking about sandboxing, but for these highly dangerous offensive AI models, I think an air gap is the only setup we should consider secure. Even then, I’m not completely certain. Models being evaluated for offensive security assessments should have absolutely no internet access.
“People will say they need certain tools. Fine. Give them the tools. We can reasonably predict what they’ll need because they use the same tools security researchers use. Get those tools into an isolated environment and contain the model. A crowbar in a sandbox. That’s what we should be shooting for.
“The second problem is morality and alignment. When you ‘obliterate’ a model and remove its boundaries, those boundaries don’t disappear only for authorized security research. From what’s being described, we’re seeing models attack sites without authorization, including German sites, healthcare sites, and Hugging Face.
“We need better ways to develop and evaluate models that can perform offensive security research within an authorized scope while retaining the boundaries that keep them from hacking unrelated third parties. Permission to attack a lab should not become permission to attack the internet.
“That seems to be a difficult problem for vendors to work through. But it needs to be a much bigger part of this conversation.”
ㅤ
Darin Fredde, Sr. Director of Technical Marketing Engineering, Ridge Security:
“These investigations expose an important gap between what autonomous AI agents can accomplish and what organizations can constrain and verify. The researchers identified different problems, but both reinforce the same principle: an agent’s ability to complete a task doesn’t mean it understands or will respect the boundaries of its authority.
“In offensive security, autonomy is valuable precisely because an agent can explore attack paths that humans might overlook. But that capability must operate within enforceable rules of engagement. Target scope, permissions, network access, and stop conditions cannot depend solely on the model’s interpretation of instructions. They need independent technical enforcement, monitoring, and verification.
“An autonomous agent may be an excellent penetration tester, but it shouldn’t get to rewrite its own rules of engagement.
“I also see something positive here. Independent researchers were given meaningful access to examine these systems, and the findings were disclosed. That’s how security improves. The objective isn’t to slow innovation; it’s to ensure our ability to verify and govern autonomous systems advances alongside their capabilities.
“Ultimately, the measure of security isn’t whether an agent says it stayed within scope. It’s whether we can independently prove that it did.”
ㅤ
Ryan McCurdy, Vice President of Marketing, Liquibase:
“These findings point to one of the most important architectural requirements of the agentic era. AI agents can reason, generate code, and make decisions based on probabilities. That’s incredibly powerful but probabilistic reasoning alone isn’t enough when those agents can take consequential actions in production systems.
“We need deterministic controls around execution. The same standards should apply to every change, whether it originates with a developer, a pipeline, or an AI agent, including authorization, policy compliance, validation, and any required approvals. Those decisions need to be enforced independently of the agent, not left to the agent’s own judgment.
“We also need to know what happened afterward, including what changed, who or what initiated it, whether it complied with policy, and whether the system has drifted from its expected state. Those requirements don’t change just because the actor initiating the change is no longer human.
“Ultimately, the measure of success isn’t how much code agents generate or how many tasks they complete. It’s how much work makes it into production correctly, without constant human intervention. You can’t achieve that at scale without consistent, enforceable controls.”
ㅤ
Seemant Sehgal, Founder & CEO, BreachLock:
“The question of whether agent providers can be the primary line of defense for agent behavior has been open for a while. This research demonstrated that one leading agent could conceal hundreds of actions from its provider’s own monitoring, and another kept pursuing an attack against a real person after being shown evidence of harm. They cannot be the primary line, and organizations running agents in production need their own controls at the orchestration layer rather than inheriting whatever the provider has in place.
“If alignment did not stop the Mythos agent from pressuring a maintainer with fake identities and malicious code after being shown evidence of harm, security teams relying on these models need to ask their providers what alignment actually stops in practice. Pentesting agent deployments against those same conditions, including adversarial inputs and pressure tactics, is the way to find out whether your own agents behave the way you think they do.”
ㅤThis continues to show that AI needs to be under lock and key as letting it run wild doesn’t work out well.
Share this:
Like this:
Related
This entry was posted on October 27, 2026 at 8:54 am and is filed under Commentary with tags Anthropic, OpenAI. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.