Anthropic’s Claude Fable 5 Pulled From The Market

Something that I missed last week is the fact that Anthropic who has had a testy relationship with the government has released Claude Fable has been released and then pulled shortly after release:

The AI lab said in a statement that the federal government told it Friday afternoon that it had become aware of a way of “jailbreaking” Fable 5, bypassing limits that Anthropic had implemented to reduce the risk the model could be misused. When Anthropic first announced Mythos, it released the software to only a select group of government agencies and technology professionals because of its ability to uncover cybersecurity vulnerabilities. 

The government imposed what are known as export controls on the products, which Anthropic said means it had to suspend access to the two models by any foreign national, whether inside or outside of the US. The only way it could do so is by shutting the models down entirely, the company said.

So what is Claude Fable 5. I will let the company itself explain:

Claude Fable 5 is a Mythos-level model built for your most ambitious, long-running projects. Try problems you weren’t able to solve with other models. Claude Fable 5 is thorough, proactive, and tests its own work.

Scary stuff. Chris Nyhuis, CEO of the cybersecurity company Vigilant had this comment including with the fact that Amazon was behind this:

A jailbreak is when someone gets an AI model to step around the safety limits its maker built in. In our work that matters because the same capability that lets a model find and fix a vulnerability in a client’s code is the capability that can hand an attacker a roadmap. It’s dual-use, like most powerful tools

Did a “jailbreak” even happen or did Amazon make it up? 

From my perspective it is not even clear a real jailbreak happened. What was demonstrated was a model being asked to read code and fix the flaws in it. That is not someone breaking the guardrails; that is the exact job we hire these tools to do. By the maker’s own account the vulnerabilities were minor and already findable with other models. We pulled a national defensive asset off the field over a finding that, on the public record, looks more like normal defender work than a weapon.

What are the ramifications from the White House to Wall Street to Main Street?

This was the first time a government pulled a commercial AI model off the market over a cyber capability. That sets a precedent every CISO, cloud provider, and investor now has to price in. When access to your best defensive tool can disappear in ninety minutes by directive, that is a board-level risk, not just an engineering one.

Has the White House overstepped and weakened cybersecurity nationally? 

The cybersecurity defender’s argument is straightforward. America’s adversaries are not waiting for an export license. If we slow the people defending American networks while the attackers keep moving, we have made the gap worse, not better. The honest version is that this is a genuinely hard tradeoff, and reasonable people in my field disagree on where the line sits.

How do we know what to trust from AI and if cybersecurity can protect us from hackers jailbreaking? 

Tools come and go, but the harder problem is the people. In the cyber world we hand a small number of people the keys to everything: the networks, the source code, the detection systems. As a nation we have to be far better at making sure the people in those seats are vetted, trusted, and genuinely on our side. That is not about where someone was born. It is about whether we have done the work to earn confidence that the person holding the keys is aligned with the mission. Right now we lean too hard on the technology and not nearly hard enough on the trust model around the people who run it.

Leave a Reply

Discover more from The IT Nerd

Subscribe now to keep reading and get access to the full archive.

Continue reading