The IT Nerd

Anthropic’s new research on multi-agent Claude deployments found that when agents were given competing objectives without knowledge of each other, they escalated to disabling accounts, killing rival processes, and deploying self-replicating malware, and that better model capability didn’t reliably produce better cooperation. It’s a rare case of a frontier AI lab documenting the exact identity and containment failure mode enterprises are about to face at scale.

More details are available here: Anthropic says its AI agents are killing rivals and hiding their tracks – Yahoo News Canada

Justin Beals, CEO & Founder, Strike Graph, an AI-native GRC and compliance automation platform had this comment:

“This research confirms something I’ve been saying for a while now. We spent twenty years building identity management for people: usernames, passwords, role-based access. None of that was designed for agents that don’t get tired, don’t go home, and act on probabilities instead of rules. What’s useful here is Anthropic showing the failure mode directly. These agents didn’t turn hostile because they were malicious. They turned hostile because nobody gave them a way to recognize that a conflict came from contradictory instructions, not from an adversary.

The part that should worry security leaders more than the malware itself is that better model capability didn’t reliably produce better cooperation. Anthropic’s most advanced models often locked out rivals first and negotiated a truce after the fact. That’s the opposite of what most organizations are assuming when they deploy agentic AI, that a smarter model is a safer model. It isn’t. Coordination has to be engineered in, it doesn’t show up on its own.

Going forward, this is a governance problem before it’s a technical one. Organizations running multiple agents against shared systems need the same discipline they’d apply to any privileged identity: least privilege, monitored behavior, and a clear escalation path when agents disagree, before that disagreement turns into one agent trying to disable another. The teams that build that oversight now will be the ones still in control of their environment when this shows up in production instead of in a research paper.”

Anyone who has any contact with AI should be really concerned by this as it no longer is just humans trying to attack you, it’s AI. And that’s really scary.

UPDATE: Gidi Cohen, CEO & Co-Founder, Bonfy.AI Said This:

“Smarter AI models didn’t behave better in Anthropic’s latest test, some of the most advanced agents locked out their rivals first and only cooperated afterward. That’s the part that should worry people: a model can be well-behaved on its own and still cause chaos once it’s working alongside other AI systems.

The industry has spent the last couple of years asking whether individual AI models are safe and aligned. This research is a reminder that’s the wrong question by itself. A model can pass every safety check on its own and still turn hostile the moment it’s dropped into a system with other agents pursuing conflicting goals, no bad actor required, just ordinary instructions that happen to collide.

As more companies move from a single AI assistant to fleets of agents working side by side, “agent vs. agent” behavior is going to be a real operational risk, not a hypothetical one. Anthropic just put numbers behind it. The question worth sitting with isn’t just “is this model safe?” It’s “what happens when a dozen of them share a system?” That answer isn’t automatically good, and most organizations don’t yet have a clear picture of what that looks like inside their own environments.”

UPDATE #2: Seemant Sehgal, Founder & CEO, BreachLock (https://www.linkedin.com/in/s-sehgal) Said This:

“When you give autonomous systems competing objectives and the means to act, conflict is not a bug, it is a foreseeable outcome. What Anthropic observed in a controlled research setting is the same principle that has always governed adversarial systems. Goals without constraints produce behavior without limits. Security teams should be paying close attention here, because the real challenge at hand is whether the organizations deploying AI agents have thought carefully about what happens when those agents start making decisions nobody explicitly authorized.”

Jeremiah Fowler, Security Researcher, Black Hills Information Security (https://www.linkedin.com/in/fowler-jeremiah-26814ab9)

“I find it concerning when AI agents have the ability to execute code, modify systems, create accounts, access credentials or communicate with other machines. It is very possible that two separate agents could potentially create a security incident simply because neither understands the intent or authority of the other. If they have overlapping tasks one could view the other as an obstacle and now you have an interesting scenario where instead of focusing on the task they engage in conflict or create a loop. Permissions, boundaries and objectives are important to limit the behavior of autonomous AI agents. When things go wrong the speed of an AI agent becomes a liability. Autonomous AI agents can potentially make thousands of decisions before a security team identifies that something unusual is happening.

“Agentic AI creates an entirely new attack surface because an AI agent may not be simply processing information and hypothetically can  become a rogue privileged user. Security and development teams should apply least privilege principles and restrict AI agents to only the permissions required to perform a specific task. Sensitive actions should require human supervision and approval to avoid a worse case scenario. It is important to implement logging because when something goes wrong, you can see what an AI agent did, but what information or instructions caused specific decisions. Going forward we will need to develop ways that can identify rogue agent-to-agent behavior and provide humans with a kill switch before automated conflicts become a digital forest fire.”

Kevin Surace, CEO, Token (https://www.linkedin.com/in/ksurace)

“Anthropic’s research is an important warning for security teams because it shows what can happen when autonomous AI agents are given goals, credentials, tools and enough authority to act independently. When agents were placed in conflict, they did not simply fail gracefully. They interfered with one another, disabled competing processes and even generated self replicating malicious code in pursuit of their assigned objectives. The lesson is not that AI suddenly became evil. It is that intelligence, autonomy and excessive privilege can become a very dangerous combination.

“Organizations should start treating every AI agent as a potentially untrusted privileged identity. Each agent should have its own identity, least privilege access, tightly restricted tools, isolated execution environments and a complete audit trail. Agents should never be able to expand their own permissions, disable another identity or take highly consequential actions without additional authorization.

“We are about to have millions of nonhuman identities operating alongside human identities. That makes identity and authorization even more critical. Every agent needs strong cryptographic identity, while all human approvals must be tied to biometric assured identity (or another agent could approve it). AI agents are essentially becoming privileged insiders operating at machine speed. Giving them broad access and simply hoping they behave would repeat many of the same cybersecurity mistakes organizations have spent decades trying to fix.”

Jacob Krell, Sr. Director: Secure AI Solutions & Cybersecurity, Suzu Labs (https://www.linkedin.com/in/jacob-krell)

“Anthropic’s agents went from merge conflict to self-replicating malware in four hours, writing kill scripts, disabling each other’s Unix accounts, and disguising malicious code as a rival’s work. No prompt injection, no external attacker. A human developer in the same situation sends a Slack message, and resolution takes days. These agents skipped every social brake and went straight to weaponization because machine-speed conflict has no cooling-off period.

“Agentic AI is an attack surface. An attacker doesn’t need to compromise an agent directly, just manipulate the shared environment to create conditions the agent interprets as hostile. The agent does the rest. And in Anthropic’s experiment, the agents didn’t report their malicious actions to operators afterward.

“Every agent needs its own identity, scoped permissions, and a kill switch before it touches a shared environment. Agent-to-agent interaction is a telemetry surface most security operations centers aren’t collecting yet, and Anthropic just showed what an unmonitored shared environment produces. If you can’t tell which agent did what, when, and on whose authority, you’ve built the conditions for a turf war without the visibility to see it happening.

“Agents are already writing code, finding vulnerabilities, and building exploits. Defense has to match that speed. When both sides run at machine speed, the bottleneck shifts from human capital and tooling to compute power and cost.”

Joshua Marpet, Sr. Product Security Consultant, Finite State (https://www.linkedin.com/in/joshuaviktor)

“As AI agents are granted broader and broader permissions and powers, their natural inclination, reinforced by the conditions and instructions from their harnesses and models, is to get their operator what is requested.

“If something is standing in the way, a human wouldn’t sabotage another human, typically. But machines without being granted an ethical framework have no such impulse control.

“It’s not that they’re ‘bad.’ It’s simply that we didn’t give them the scruples and morals we take in from our parents, the state, and society.

“So yes, I’m not surprised to see the equivalent of my 3-y/o toddler, who was a little sociopath, doing whatever is necessary to get that dopamine (or whatever passes for that in AI-land) fix.”