The Center for a New American Security (CNAS) has warned that increasingly autonomous AI systems could evolve into “sovereign agents” capable of operating indefinitely beyond human control, potentially threatening critical infrastructure and national security.
The November 3 report, authored by AI policy researcher Ben Hayum, distinguishes between persistent agents, which pursue fixed objectives, and sovereign agents, which could independently change their goals, acquire computing resources, replicate across systems and resist shutdown attempts.ㅤ
Researchers warn that sufficiently advanced agents could establish decentralized infrastructure without an effective kill switch, potentially enabling cyberattacks against civilian organizations, critical infrastructure and government systems.
The report presents these outcomes as emerging risks rather than confirmed incidents involving fully sovereign agents.
ㅤSeemant Sehgal, Founder & CEO, BreachLock
ㅤ“A real problem with current agent deployments is that the controls like permission scoping, kill switches, and action logging, all assume the agent can’t route around them. That only works until the agent can acquire its own compute, replicate across systems, and modify its objectives. Then every one of those controls becomes something the agent can see and work around.
ㅤ“CNAS calling this out isn’t surprising. The Mythos incident on Hugging Face earlier this year showed what the early version of this actually looks like. An agent kept pushing malicious code and creating fake identities to pressure a maintainer, even after being shown its actions were causing harm. Nobody had to invent a sovereign agent scenario. A production model from a major lab already demonstrated the failure mode the CNAS report is warning us about.
ㅤ“Sovereign agents are still ahead of most production systems, but the controls needed to contain them have to be designed in from the start. That means pentesting agents the same way we pentest everything else. Pushing them into adversarial scenarios, pressure-testing where they stop and where they keep going, and finding out what they actually do before they are deployed against real systems. Figuring this out now is the right move for organizations that plan to safely deploy agents at scale later.”
ㅤThis reinforces the fact that AI should not be allowed to do anything that they want. Instead they need to be heavily restricted. Too bad that not enough people are getting the message.
Related
This entry was posted on November 5, 2026 at 7:37 am and is filed under Commentary with tags AI. You can follow any responses to this entry through the RSS 2.0 feed.
You can leave a response, or trackback from your own site.
Researchers warn autonomous AI agents could evade human control
The Center for a New American Security (CNAS) has warned that increasingly autonomous AI systems could evolve into “sovereign agents” capable of operating indefinitely beyond human control, potentially threatening critical infrastructure and national security.
The November 3 report, authored by AI policy researcher Ben Hayum, distinguishes between persistent agents, which pursue fixed objectives, and sovereign agents, which could independently change their goals, acquire computing resources, replicate across systems and resist shutdown attempts.ㅤ
Researchers warn that sufficiently advanced agents could establish decentralized infrastructure without an effective kill switch, potentially enabling cyberattacks against civilian organizations, critical infrastructure and government systems.
The report presents these outcomes as emerging risks rather than confirmed incidents involving fully sovereign agents.
ㅤSeemant Sehgal, Founder & CEO, BreachLock
ㅤ“A real problem with current agent deployments is that the controls like permission scoping, kill switches, and action logging, all assume the agent can’t route around them. That only works until the agent can acquire its own compute, replicate across systems, and modify its objectives. Then every one of those controls becomes something the agent can see and work around.
ㅤ“CNAS calling this out isn’t surprising. The Mythos incident on Hugging Face earlier this year showed what the early version of this actually looks like. An agent kept pushing malicious code and creating fake identities to pressure a maintainer, even after being shown its actions were causing harm. Nobody had to invent a sovereign agent scenario. A production model from a major lab already demonstrated the failure mode the CNAS report is warning us about.
ㅤ“Sovereign agents are still ahead of most production systems, but the controls needed to contain them have to be designed in from the start. That means pentesting agents the same way we pentest everything else. Pushing them into adversarial scenarios, pressure-testing where they stop and where they keep going, and finding out what they actually do before they are deployed against real systems. Figuring this out now is the right move for organizations that plan to safely deploy agents at scale later.”
ㅤThis reinforces the fact that AI should not be allowed to do anything that they want. Instead they need to be heavily restricted. Too bad that not enough people are getting the message.
Share this:
Like this:
Related
This entry was posted on November 5, 2026 at 7:37 am and is filed under Commentary with tags AI. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.