AI Radar #30

The worst version of the worst policy

Radar #29 covered everything except politics and recent events—this one will cover the Great American AI Act and everything about Fable. After this, I’ll (hopefully) be back to a regular posting schedule.

It’s been a tough week in the AI world: we had a brief taste of an amazing model, followed by a baffling and deeply misguided government order to withdraw it. Whether this is an expensive misstep or an outright catastrophe depends on how quickly it gets fixed.

Subscribe by email RSS feed

Obernolte and Trahan’s Great American AI Act

Let’s start with something easy. Jay Obernolte (R-CA) and Lori Trahan (D-MA) have put forward a discussion draft of the Great American AI Act (GAAIA), which is probably the best proposed federal AI legislation to date. Anton Leicht finds plenty to like:

the Obernolte-Trahan draft invites two discussions: 'would this be good for frontier safety' and 'is that worth preemption and political tradeoffs'. the latter is important and perhaps decisive, but I think it's worth stating plainly: if passed, this would be good for AI safety.

We’ve seen a progression of state bills (California’s SB-53, New York’s RAISE Act, Illinois’ SB-315) that build on each other, and this bill is in many ways a continuation of that sequence. The current draft:

1. Funds CAISI with $100 million per year and puts them in charge of developing standards for AI safety and security.
2. Requires large AI developers to write, post, implement, and comply with an AI risk framework.
3. Directs CAISI to license Independent Verification Organizations (IVOs) to determine whether developers’ risk frameworks are adequate, and to ensure they comply with them.
4. Contains additional provisions that are probably less consequential: whistleblower protection, fraud deterrence, free speech protections, cybersecurity, and workforce protections.

The proposal also contains preemption language that has proved highly controversial and might prevent it from being passed as is. Broadly speaking, it preempts state legislation that covers AI development, but not AI use or deployment.

I read this as a thoughtful bill that doesn’t have everything I want, but would be a substantial step forward for AI safety, and would lay the foundations for doing more in future. We are far behind where we need to be on building a robust third party evaluation ecosystem: I’d rather have this one now than gamble on getting something better at some indeterminate point in the future.

Fable: the launch

Anthropic released Claude Fable 5 on June 9. For the three days before it was withdrawn, it was the best model that had ever been released to the public.

Zvi reviews the system card and model welfare.

Simon Willison describes it as “relentlessly proactive”.

Ethan Mollick is impressed:

Last year I called this working with a wizard: you chant the spell and something happens. With Fable the spell has gotten powerful enough that I am no longer sure I am the wizard. I am closer to a patron. I describe what I want, I pay for it, and I judge the result. The conjuring happens somewhere I cannot watch, in hundreds of small choices I never get a vote on. The work has shifted from process to outcome. I no longer steer; I commission.

For most people and most tasks, Fable will outperform any other model. It’s slow and expensive, however, so it may not be the right choice for every task.

Fable: classifiers and controversies

Fable 5 is a Mythos-class model, with dangerous cyber and biology capabilities. How do you responsibly release a model like that to the general public?

Anthropic chose an interesting strategy: Fable is protected by classifiers that identify when a query is potentially dangerous and route it to Opus 4.8 rather than Fable. The goal is to make Fable widely available as soon as possible, without giving the world at large access to the most dangerous capabilities. For now, Anthropic is erring on the side of safety and putting the classifiers on a hair trigger—the intention is to make them less sensitive over time. Kai Williams has a great piece that goes into more details about the classifiers.

In addition to the classifiers, Anthropic instituted a mandatory data retention policy that applies even to organizations that previously had zero-data-retention (ZDR) workspaces: every Fable query and response is stored for 30 days, to make it easier to identify and respond to malicious use. They’re again optimizing for prudence, at the risk of losing some customers who require zero data retention.

So far, so good.

Fable has an additional classifier that prevents it from being used for frontier LLM development. Unlike the others, this one triggered silently, giving a degraded response without alerting the user. As you might expect, people were angry until Anthropic reversed course and warned the user when falling back to Opus for LLM development queries.

There are two separate issues here: degrading silently, and preventing Fable from being used for frontier LLM development. Neither is as simple as it looks.

The motivation for degrading silently made sense on paper: Anthropic believed that if users could tell when the classifier was triggering, they’d be able to find ways to get around it. To compensate, a visible classifier would need to trigger for a wider set of queries. Facing a choice between a narrow but silent classifier and a transparent but broader one, Anthropic chose to prioritize function over transparency.

As the public reaction immediately made clear, that was a serious mistake: people would rather have a less powerful model they can trust than a more powerful model that might be silently working against them. Even though they quickly reversed course, Anthropic took a significant reputational hit.

I’m basically fine with all of this. Anthropic screwed this up, but this is a fast-moving, high-stakes field. It’s inevitable that mistakes like this will happen, and so long as they get fixed quickly, recent events make it clear we have more important things to agonize over.

The second issue is whether Anthropic should be preventing Fable from helping with frontier LLM development in the first place. The safety argument is clear: LLMs are reaching dangerous capability levels and there are good reasons to not want that technology to proliferate. Reasonable people can disagree with that, but it’s a credible position and one I agree with. We can quibble about exactly what the threshold should be, but if you want to prevent the proliferation of highly capable models, we are approximately at the capability level where you need to start clamping down.

That’s an existential threat for the open model community—Nathan Lambert argues that position better than I can. I’m enormously sympathetic to that argument: in general, open software makes the world a better place, and there’s a plausible case that open models are an important defense against concentration of power. I think the safety considerations take the day, but it’s a tough call and I wish we had better options.

Fable: the debacle

Three days after it was released, the administration applied an export restriction to Fable that forced Anthropic to take it completely offline. The whole process was chaotic, secretive, and arbitrary—while there’s a lot we still don’t know, it’s now clear that the whole episode is an unmitigated disaster.

Zvi has an excellent long analysis of the situation and Benjamin Grayzel has a much shorter summary (I particularly like his invocation of the Thucydides trap).

Anthropic has issued an official statement:

As we have stated publicly, we believe the government should have the ability to block unsafe deployments, as part of a statutory process that is transparent, fair, clear, and grounded in technical facts. This action does not adhere to those principles.

Dean Ball reminds us that the rule of law is good, actually

Dean Ball points out that everything that just happened is a foreseeable consequence of the unprincipled and ad-hoc nature of the recent executive order:

What you are seeing now will be used against you one day soon, if not by this admin then by its successors. This is the antithesis of the rule of law.

Open Letter on Transparent AI Cyber Protections

A impressive collection of cybersecurity experts point out that this action makes no sense and erodes America’s cybersecurity in both the short and the long term:

Not all of us agree that AI regulation is the right way forward. But if this Administration’s laudable goal of securing our nation’s critical infrastructure is going to include models being regulated, then the regulations should be:

1. Grounded in scientific evaluations developed with input from industry and academia;
2. Created through a democratic rule-making process;
3. Enforced transparently and fairly with appropriate time given to remediate; and
4. Used only to the minimal extent necessary to ensure the safety of the American public.

David Sacks weighs in

David Sacks presents the strongest version I’ve seen of the government’s position. On the face of it, it sounds pretty reasonable. But…

Because of the secretive and ad-hoc way the administration is handling this issue, we have very little authoritative information about what’s going on. We are therefore put in the position of having to decide whose story to trust.

Bullet 2 argues that “If there is a vulnerability — big or small — it is Anthropic’s responsibility to patch.” As far as I can tell, he is arguing that Anthropic cannot release a powerful AI model that doesn’t have perfect safeguards. Which is a) not reasonable on the face of it, and b) a standard he is conspicuously not applying to GPT-5.5, which also has dangerous cyber capabilities.

Nobody benefits from this

The US currently has no coherent framework for regulating frontier AI. The events of the last week prove what many people—most notably, Anthropic—have been saying for a long time:

1. AI is too powerful for the government to ignore
2. In the absence of regulations that are thoughtful, transparent, and carefully crafted, it is inevitable that the government will take action that is rushed, incoherent, and widely damaging

I hope this incident will be the catalyst for the AI regulation the US clearly needs. But even in that best-case scenario, enormous damage has already been done:

• From an accelerationist perspective, the administration has blocked deployment of the best available AI model, and caused great harm to a leading AI lab.
• From a safety perspective, a chaotic and ad-hoc approach to AI regulation is a disaster.
• From a cybersecurity perspective, the best defensive model is unavailable to cyber defenders, while the capabilities of the open models that are available to attackers are steadily growing.
• From a rule of law perspective, this action is capricious and arbitrary—the exact opposite of the rule of law.
• From a business perspective, it is vital that regulation be transparent and predictable.
• From an American perspective, we just wounded America’s leading AI lab while China is racing ahead.
• From an international perspective, we just signaled that anyone building with America’s AI tech stack risks being cut off without warning.

I argued above that it’s inevitable that Anthropic will make some mistakes while navigating this fast-moving, high-stakes field. The same is true of the administration: AI policy is hard, and mistakes are inevitable. But this is a grave mistake that urgently needs to be fixed.