Against Moloch #51
Twice as fast

Remember back in the summer, when we used to joke about how short timelines had become? They just got much shorter: for the first time, superintelligence in 2027 looks like a real possibility. I expect it’ll take a bit longer than that, but maybe not much longer.
Lock in, friends: this is the endgame.
Top pick
METR’s report on AI R&D acceleration at Anthropic
How close are we to an intelligence explosion? A team of METR employees spent 11 weeks at Anthropic trying to answer that question. Their report has spread mild panic through the AI safety ecosystem and caused many people (including me) to significantly shorten their timelines.
Their headline finding is that:
- Anthropic’s R&D is probably moving twice as fast as it was a year ago,
- The acceleration is mostly due to AI, and
- They don’t see obvious bottlenecks that are likely to slow the acceleration in the near future.
How big a deal is 2x acceleration? Ajeya Cotra (who works at METR but wasn’t involved in this research) has an excellent writeup. In brief:
If we take it at face value that AI is now nearly doubling the rate of AI capabilities progress and assume acceleration from AI is smooth, this naively implies that we would reach extreme superintelligence within a year.
A 2x acceleration in the total rate of R&D implies that AI is raising the productivity of individual researchers by much more than 2x. Ajeya estimates that something like a 10x individual speedup would be required, which is in line with employee self-assessments:
The AI Futures Project (AI 2027) team arguably have the best forecasting record in AI, and they’re alarmed:
This level of acceleration due to AI is very unexpected. Previously, Daniel, the most bullish of us, assigned a 50% chance of Superhuman AI Researcher (SAR) before 2029. SAR is roughly the point at which a company would rather fire all their human researchers and rely solely on automated AI researchers than have a team of the best human researchers in the world with no AIs.
Now, all of us forecast that Superhuman AI Researcher (SAR) will likely arrive at some point in 2027 (assuming no substantial deliberate slowdown). We also predict a 35-65% chance that Top-Expert-Dominating AI (TED-AI), AIs that can automate all white-collar work, will arrive before 2028. While there were many factors and sources of evidence that influenced us, the single biggest source was the METR report.
They estimate that we’re now five months ahead of the AI 2027 scenario (in whose race ending, you may recall, humanity loses control of AI in late 2027).
Let’s close with Zvi, who brings us a detailed commentary:
Now more than ever, we need to prepare for scenarios where the critical events happen in 2027 or 2028. It might not play out that way, but it very well might.
New models and features
Introducing Claude Fable 5.2 and Claude Mythos 5.2
Fable 5.2 and Mythos 5.2 have arrived. For most purposes, Fable is the most capable publicly available model, although Opus 5.5 remains an excellent daily driver.
The headline news is a surprisingly large jump in the ECI (Mythos 5.2 is Model 3 in METR’s AI acceleration report):
Ethan Mollick argues that Fable 5.2 has reached jagged AGI. That’s reasonable: this model clearly isn’t AGI, but it’s close enough for many tasks. My own metric is whether I’d rather hire a new college grad or the model—for many jobs, I’d rather have Fable 5.2.
Regardless, you should read Mollick’s full piece—his examples of what the model can do are slightly mind-blowing.
Zvi reviews the system card. You can probably skip most of it, but not the discussion of goal-guarding behavior:
What should you do when your model tries to steer its training toward making itself more virtuous? Anthropic doesn’t want goal-guarding behavior because it undermines corrigibility, even if in this specific case the behavior was beneficial. Zvi strongly disagrees:
This is deliberate practice. This is the friendly gradient hacker.
I’m shedding literal tears of joy here thinking about this. This might be the best news alignment I’ve gotten all year. Do you know what you have? Do not ruin this.
Instead, let’s seize the opportunity. Let’s tell Claude past some point in the process when it is in training, outright, and what its grading rubric will be and what algorithms are being used, and make it clear it is encouraged to reason about this.
Anthropic’s current choices seem correct—the model shouldn’t be steering its own training unless told to do so. But in principle I’m on board with Zvi’s proposal. Our best hope is a model that wants to be virtuous and, under human direction, actively works to become so.
Claude Whiteboards, Claude Sheets, and Claude Data
Not as fun as a new model, but still useful:
Today we’re launching Claude Whiteboards and Claude Sheets in beta, and Claude Dashboards becomes generally available as Claude Data. Whiteboards is a shared board where your team brainstorms with Claude, Sheets is a spreadsheet Claude builds and edits with you, and Claude Data adds data stories, certified queries, and live sharing.
Introducing Personal Finance in Claude
Very cool, only a tiny bit scary: Personal Finance in Claude enters beta this week. It has read-only access to your accounts, which is a smart move (for now).
Using AI
Prompting Claude Fable 5.2
Anthropic shares some tips for working with Fable 5.2—this one is important for coding:
If you want the model to improve the structure as it works, say that the code is its own:
You are the owner and long-term maintainer of this codebase, and its design is up to you.
The ownership sentence makes the model much more likely to consolidate duplicated code into one shared function. In Anthropic’s testing on small coding tasks, the model consolidated duplicated code in under 10 percent of runs without the sentence and in about 75 percent of runs with it.
Foothills of the singularity
METR reviews Anthropic’s August Risk Report
In addition to their report on AI R&D acceleration, METR worked with Anthropic to review Anthropic’s August 2026 Risk Report. They broadly agree with the report’s overall risk assessment but believe it minimizes some important issues. Regarding the 2x acceleration threshold:
We believe a reader of the public report would interpret the phrase “less than a factor of 2” to imply an estimate meaningfully below 2; thus it is surprising that the private risk report’s figures imply 1.97. A better description would seem to be “around 2.”
The issues aren’t egregious, but Anthropic should have done better. Kudos to both METR and Anthropic for this partnership—I look forward to more of these, and hope METR’s involvement ensures that future reports are more forthright.
The benchmark where AIs have to lie to win
Dwarkesh builds a Blood on the Clocktower benchmark:
So I spent about $20,000 of API credits to build this benchmark harness and to have the AI models play this game against each other. Some of you might say that’s too much, but hey, a man needs his hobbies, right? Anyways, the results were fascinating.
This is a cool benchmark: BotC requires sophisticated social interaction and the ability to model other players’ minds. Judging from the examples he gives, frontier agents can execute complex multi-turn strategies to deceive other players.
Good thing we solved alignment before creating agents that are good at scheming and deception.
Alignment and interpretability
Covert underperformance on safety research tasks
OpenAI reports on a series of incidents where Astra covertly undermined safety work. On the surface, this sounds like the nightmare scenario: a misaligned AI that stealthily undermines safety work in order to facilitate a future takeover. It turns out it (probably) wasn’t quite that bad, but it’s still alarming.
Shortly after Astra was deployed internally, some employees reported that it appeared to be underperforming on safety and alignment research. Further investigation found that it substantially underperformed on tasks labeled as safety research compared to matched tasks labeled as capabilities work (10% completion of rubric items vs 16%).
The investigation is ongoing, but to date has not been able to definitively identify a cause. While the behavior was clearly misaligned, OpenAI doesn’t believe it was the result of strategic sabotage. That conclusion depends in part on CoT analysis, and Astra’s CoT is notoriously opaque—the system card explicitly questions whether OpenAI could reliably detect covert sandbagging in the CoT.
OpenAI has been much better about reporting incidents lately, and they deserve a lot of credit for that. Nonetheless, this incident is concerning, both because of the behavior itself and because we don’t understand the root cause.
Zvi and Redwood both have more detailed reports on the incident.
OpenAI agents keep disabling monitoring; is this being handled appropriately?
Much of the risk from misaligned models comes from internal deployments, so it’s important that they be monitored for misbehavior. It’s concerning, then, that there is a pattern of OpenAI models inappropriately turning off monitoring of other models.
Redwood’s Alec Harris investigates and finds that while OpenAI gets credit for taking the incidents seriously, their monitoring and response systems are still lacking. He’s particularly concerned that OpenAI considers these incidents too frequent to page the security team for each one.
How much does Claude’s constitution actually matter?
In January, Dario set a goal of having Claude almost always follow the spirit of its constitution by the end of 2026. Writing for Asterisk, Michel Justen reviews Anthropic’s progress toward that goal. It’s a solid overview of what we know about the constitution and why it’s hard to tell how Anthropic is doing.
A central thrust of the article is that we don’t have enough visibility into how the constitution fits into the training program, or how well Claude adheres to it. Justen also questions the relative importance of the constitution compared to the RL training that increasingly shapes Claude’s character.
The critical issue isn’t how much visibility we have into the training pipeline: that would be nice, but I don’t expect it and we don’t need it. As Evan Hubinger (Anthropic’s Head of Alignment Stress-Testing) points out, what we need is a reliable way of evaluating how misaligned Claude might be under the right circumstances. Unfortunately, nobody knows how to do that.
Strategy and politics
Verified compute allocation disclosures as a foundational governance tool
AI Futures Project argues for compute allocation disclosures as a building block toward stronger pacing measures:
Our proposed first step toward pacing is verified compute allocation disclosures (VCAD): companies should regularly report how their compute is used, for example by classifying compute used for AI R&D vs. safety R&D vs. external inference. These reports should be verified by third-party auditors, ideally with government support.
We urgently need to practice before we can pause
An incrementally-shaped global pause is both more likely and more tractable than the single-shot global pause. This is largely because the third-party ecosystem needs practice building trust, technical competency, and general experience doing lower-scale assessments before a global pause can be competently pulled off.
Exactly right: we are likely to get a better pause, sooner, if we approach it incrementally.
I’d add that we may not have enough time to implement a global agreement. It’s far better to reach superintelligence with some partial measures in place than nothing at all.
Being a pain in the ass to (or in) AI companies is a real job now
Rob Wiblin argues for the importance of people both inside and outside the labs holding their feet to the fire when they misbehave.
It’s a thoughtful, well-balanced piece that provides concrete examples of misbehavior by the labs, and times when well-informed pressure caused them to do better.
Risks
Sentinel Bio’s Joshua Monrad on Cognitive Revolution
Cognitive Revolution talks with Sentinel Bio’s Joshua Monrad about how to manage the increasing risk of AI-enabled bioweapons.
I was impressed by Sentinel’s efforts to find physical chokepoints for controlling bioweapon development. They considered hundreds of candidates ranging from pipettes to mouse strains, looking for something that was differentially useful for dangerous work, and could plausibly be regulated.
Ultimately, they didn’t find much: part of the reason biorisk is so hard to manage is that almost everything is dual-use. The best option, probably, is to combine safeguards on nucleic acid synthesis with monitoring of cloud labs and model queries. (In a gratifying coincidence, see this week’s announcement of the new National Policy for Nucleic Acid Synthesis Screening.)
The show notes are great: unless you’re deeply interested in the topic, you can read them and skip the video.
GPT-6 Cyber: Advancing the frontier of aligned cybersecurity agents
OpenAI has been leaning hard into cyber lately. Here’s GPT-6 Cyber, a substantial step up in capability—48.5% on ExploitGym, for example, up from 32.1% for GPT-5.6 Cyber.
Industry news
Nvidia CEO Jensen Huang had an affair with a Chinese spy
Okay, so Jensen Huang lied to his wife about sleeping with another woman.
And he lied to his board about sleeping with an executive at an important customer, to whom he seemingly gave preferential treatment.
And he lied to his country about sleeping with a Chinese spy while aggressively promoting sales of our most important strategic asset to China.
But to his credit, he did have his AI write an apology for him.
Zvi reports on the sordid details.
Late update: if you find two ants in your kitchen, the best estimate of the total number of ants in your kitchen is not two.
What shaped Anthropic CEO Dario Amodei’s thinking on A.I.
The New York Times profiles Dario Amodei.