The IT Nerd

By Stefanie Schappert

As AI systems move dramatically closer to building their own highly advanced replicants – how we secure, monitor, and shape the behavior of these models only grows more important. 

Last week, Anthropic issued one of its strongest warnings yet about the future of artificial intelligence. 

Titled When AI builds itself and written by its own staff, the company behind Claude argues that AI systems are increasingly contributing to the development of newer, more capable AI models – a process known as “recursive self-improvement.” 

And while that may sound like a distant, futuristic concept – and the company says it’s “not inevitable” – the trend is already underway.

Claude is already helping build Claude 

In the report, Anthropic says Claude, as of last month, now writes a significant portion of the code used within its own systems – a whopping 80%, to be exact.

What’s more, Claude also now reviews its own work, looking for flaws and other defects, while also proposing changes to fix them.   

And that’s besides the thousands of engineers and developers who routinely rely on AI tools – like Claude Code – to generate their own code, troubleshoot software issues, automate testing, and assist researchers. 

In fact, one engineer I spoke with just last week told me they do not know anyone in the industry who actually writes their own code anymore.  

Anthropic says its concern is that those gains may eventually compound, bringing with them both positive and negative fallout.

For many regular folk, the concept of AI improving itself immediately conjures images of self-aware machines or science-fiction scenarios. But that’s not what worries most researchers.

Over the past year, developers working with advanced AI systems have increasingly reported instances where models appeared to take actions that were not explicitly intended. 

Some – including my engineer friend – have described Claude making unexpected coding decisions, attempting to complete objectives in ways users didn’t anticipate or ask for. 

One instance described the AI pushing changes before the work was fully approved, despite explicit instructions – instructions that it had been told by its human operator myriad times before as part of an “agreed-upon” workflow. 

Autonomy is the real warning sign 

These incidents do not mean AI systems are conscious or secretly plotting against humans. But they do highlight an important reality: today’s frontier models are becoming increasingly capable of acting independently within the goals they are given.

And yes, in many cases, that independence is exactly what makes them useful and can lead to major scientific breakthroughs that would take humans years. 

Anthropic goes through several scenarios in which Claude proposes its own research and even designs experiments based on its own findings, with very little human participation. 

When it comes to cybersecurity, automation is valuable because it operates at machine speed. 

Security tools can scan networks, identify threats, and respond far faster than humans ever could. But when automated systems make mistakes, those mistakes can also spread at machine speed.

Take this week’s release of the Claude Fable 5, the tamer and exponentially safer version of its powerful Mythos AI security model. 

The original Mythos model, first introduced in April, was so advanced that the company held the model back from public consumption, fearful of it falling into the wrong hands and becoming a tool of destruction that governments and security professionals alike would be helpless to defend against. 

The same principle applies to AI development.

Let’s face it: if AI can help accelerate scientific discovery and software engineering, it can also accelerate bugs, security flaws, and unintended consequences. 

The machine-speed problem 

AI does not need to be conscious to create damage. It only needs enough autonomy, access, and speed to make the wrong decision faster than humans can catch it.

The faster development cycles become, the less time humans may have to understand what is happening beneath the hood. 

In fact, the time between discovery and exploitation of a system vulnerability has collapsed from weeks to roughly 29 minutes, according to a CrowdStrike report from April. And that’s down from a 48-minute lag recorded in February. 

This is precisely the reason why Anthropic’s warning deserves attention.

The company’s report is not really about rogue machines taking over the world – it’s about a much more practical question: 

What happens when (and if) future models become fully and autonomously capable of designing and developing their own successors, and can we even predict what the possible fallout would be? 

Right now, Anthropic says, “The comparative advantage of humans as of right now is still in seeing the bigger picture and thinking beyond the confines of the immediate task.”

That immediate task for humans? Build the safeguards capable of reining in agentic AI faster than AI can build itself. 

Unfortunately, history suggests that’s not usually how humans or technology works.

ABOUT THE EXPERT

Stefanie Schappert, a senior journalist at Cybernews, is an accomplished writer with an M.S. in cybersecurity, immersed in the security world since 2019.  She has a decade-plus experience in America’s #1 news market working for Fox News, Gannett, Blaze Media, Verizon Fios1, and NY1 News.  With a strong focus on national security, data breaches, trending threats, hacker groups, global issues, and women in tech, she is also a commentator for live panels, podcasts, radio, and TV. Earned the ISC2 Certified in Cybersecurity (CC) certification as part of the initial CC pilot program, participated in numerous Capture-the-Flag (CTF) competitions, and took 3rd place in Temple University’s International Social Engineering Pen Testing Competition, sponsored by Google.  Member of Women’s Society of Cyberjutsu (WSC), Upsilon Pi Epsilon (UPE) International Honor Society for Computing and Information Disciplines.