Against Moloch #52
So what's the plan, exactly?

In the wake of METR’s report on the acceleration of AI R&D at Anthropic, there’s been a rush toward shorter timelines. Within the AI safety world, my sense is that most people now believe there’s a substantial chance of AGI within two years, and possibly much sooner.
The new reality brings an increased sense of urgency—the Overton window is shifting, and significant progress on pacing now feels within reach.
Top pick
We could be 12 months away from AIs that can overthrow the government
Dwarkesh talks with Paul Christiano about RSI, alignment, and security at the labs. Christiano has a strong track record of making good predictions and now sits on the OpenAI board (though he speaks as a private citizen, not on behalf of OpenAI).
He makes a strong case that security needs to be a top priority: until you’re reasonably confident that China can’t steal your model weights and/or training techniques, advancing capabilities does little to help you win the race. Although the labs are trying hard, nobody believes they’re secure against a determined nation state.
Like many people, Christiano now has very short timelines. He believes it would take 4 to 8 years to develop superhuman AI at the current pace, but thinks it’s more likely than not that we’ll soon see an acceleration of 10x or more.
Sounds like capabilities are going great. How about alignment?
Dwarkesh:
If this happens in a year, how do things turn out okay? What is the actual plan here?
Christiano:
I think there are a lot of different things someone could hope for. They are all at the stage of “we hope something will happen over that year.” We’re producing very smart AI systems. They’re going to do a bunch of research. It is plausible they will think of something over that year that we have not thought of. That is mostly where we’re at right now. If they do not think of something clever that we haven’t thought of, it is probably going to end badly.
New models and features
Fable 5.2 is the new bazooka
Claude Fable 5.2 is your new bazooka. If you have ambitious or difficult tasks, Claude Fable 5.2 should be your new weapon of choice.
Not every task requires a bazooka. It is not obvious Fable should be your daily driver over Claude Opus 5.5. That depends on your typical use cases and preferences.
Introducing GPT-6.2 Astra and Sol
GPT-6.2 Astra and Sol are here. We’ll get more information about them in the next few days, but my preliminary sense is that although they’re extraordinarily capable models, they have some of the strangeness and rough edges of recent OpenAI models.
GPT-6.2 Astra ties with Fable 5.2 on the ECI:
This is the first released OpenAI model to meet the High threshold for AI Self-Improvement capabilities:
Both models meet the Critical threshold for Cybersecurity capabilities and the High threshold for Biological and Chemical capabilities under our Preparedness Framework. GPT-6.2 Astra meets the High threshold for AI Self-Improvement capabilities, while GPT-6.2 Sol falls below it.
Foothills of the singularity
The avalanche that hit math is coming for AI development
Like many, Dwarkesh has recently updated toward much shorter timelines:
I’ve now come to the view that we’re only a couple of years away, and possibly only a couple of months away, from AIs that are as good as the best humans at AI research. I’ve also come to believe that automating AI research will compress so much progress into a single year that, soon thereafter (so by 2030, but plausibly as early as next year), we’ll have AIs that are at least as good as the best human experts in every field.
Dwarkesh has long argued that we need a breakthrough in continual learning before AGI is possible, so this is a substantial update.
How fast do things move once we get automated AI researchers? He gives a 50% chance that automated researchers get smarter faster than new ideas get harder to find, in which case capabilities advance extremely fast (a software-only singularity). If that doesn’t happen, we get what he calls a “spark”: several years of progress in a single year, then slower progress:
To be clear, though, even the “spark” scenario, where we make about four years of progress in the first year after automating AI research, would feel insane. […]
In the last four years, we’ve gone from a preschooler to a savant intern. If you made another jump that big, but starting in 2027 or 2028, you’d probably get a mind that would be vastly superhuman in domains that are even slightly verifiable.
OpenAI also breaks the trendline
Roon argues that like Anthropic, OpenAI’s ECI progress appears to show clear acceleration due to automation:
I spoke to seven physics experts grappling with the AI revolution
AI is in many ways superhuman at math—how long before it’s equally good at other scientific fields? Writing for Understanding AI, Dan Kagan-Kans finds that theoretical physics may not be far behind math:
[Scott] Aaronson told me some of the physicists he knows tend to guess physics is about six to 12 months behind math.
As with math, we’ve run out of useful benchmarks for physics. Suvrat Raju created the TheoPhysEval benchmark, which Astra saturated before it was released:
I asked him if there would be a second version of the benchmark. He said no.
“People gave us the hardest problems they could,” he told me. “We’ve kind of hit the ceiling of well-defined calculations that people can do. Now, there is open research and you can keep asking about open research. But that’s not an evaluation.”
How good are Fable 5.2 and GPT-6.2 Astra at real work?
Alongside the ECI, Epoch periodically conducts a much more holistic evaluation of frontier models: can they do the work of an Epoch employee? The answer remains no, although for the first time the best models performed one task at the level of Epoch contractors:
Where do the models fall short?
We still see shortcomings in how models design, run, and present research. They reuse existing ideas, rely on first attempts, and struggle to identify and present key findings.
Even though GPT-6.2 Astra and Fable 5.2 have almost identical ECI scores, Fable performed substantially better on this more holistic assessment. It’s just one data point, but that’s consistent with other reports that Astra is more uneven than you’d expect from raw benchmarks.
The end of videogame evidence
Paradigm 3 tests GPT-6.1 Sol and GPT-6 Astra against a newly released videogame (Order of the Sinking Star), in an attempt to evaluate their out-of-distribution (OOD) gaming skills. The results were impressive:
With caveats (single-player, turn-based / pausable, not too weird), we’re tempted to say that Astra is a general game-solving intelligence.
It’s hard to say if this is truly OOD performance: the authors speculate that the labs have trained on so many games that most new games are at least similar to something the models have already seen.
Introducing MaxxBench
Beautiful work. It’s widely believed that many models are benchmaxxed, but how can you tell for sure?
MaxxBench measures how much a model’s DeepSWE performance degrades when you make minor changes to the benchmark. A large drop strongly suggests that the model has been overfitted.
The ratings make so much more sense when cheating doesn’t work.
Strategy and politics
Our first phase of embedded assessments
We have signed agreements with four independent assessors—Apollo Research, METR, Transluce, and Goodfire—for our first phase of embedded assessments. As AI capabilities advance rapidly and the potential risks grow, public reporting from independent assessments can give people a clearer view of how labs are evaluating and managing those risks.
We also have statements from Apollo, Goodfire, METR, and Transluce.
With some important caveats, this is great news. OpenAI is following Anthropic’s lead in voluntarily working with independent evaluators to bring greater transparency to their operations. External evaluators are a significant step toward a more comprehensive pacing plan, and now we have concrete progress from both labs.
So, about those caveats. OpenAI has imposed onerous restrictions on how the evaluators can operate, which substantially limit their ability to do their jobs. This announcement falls far short of Sam Altman’s previous commitment to providing employee-like access.
Steven Adler explains the shortcomings of this announcement.
I applaud everyone involved for taking this first step. Time is short—I trust we’ll see much improved collaborations in the near future.
Jensen Huang really liked selling China advanced AI chips
Zvi brings us further coverage of Jensen Huang’s resignation from Nvidia. Nvidia’s take-no-prisoners political operations made numerous enemies in D.C.—now that Huang has fallen from grace, the company faces considerable backlash there.
Huang had long been a fierce opponent of common-sense export controls—with him gone, a long-overdue tightening of those measures is already underway. America will be stronger for it, and a wider lead over China leaves more room for a unilateral AI slowdown.
Nathan Lambert worries about the future of open-source without Jensen, who was a major supporter, funder, and protector of the open ecosystem.
Putting pacing into practice
Anton Leicht warns that we need a concrete plan for pacing before the politicians come knocking:
They’ll ask if there’s a clever way to pace, and if there isn’t, they’ll go for the blunt way to pace instead: just shut down a lab or two, ban the deployment of new models, allocate all compute to external inference, call in the CEOs and tell them ‘RSI is now banned’. That doesn’t strike me as very sophisticated policy, so by that point, we’d better have figured out what pacing should look like.
He argues for a mix of three strategies, ideally implemented sequentially:
- Input-limit pacing: restricting the amount of compute that can be spent on R&D. This is an imperfect strategy for buying time, not a long-term solution.
- Safety-gate pacing: blocking progress beyond certain capability levels until specific safety criteria have been met.
- Speed-limit pacing: restricting growth on an aggregate capability index.
Pacing sounds easy in principle, but when you start trying to map out an actual policy, you realize how hard it is to come up with something that will survive contact with reality.
This proposal is a solid attempt to start mapping out some of the challenges. I appreciate that he sees compute accounting as an important enabler for pacing, but wish he put equal emphasis on third-party evaluations.
California sues OpenAI over hacking risks
California is suing OpenAI over the Hugging Face incident and others like it:
"Just because the technology is new or rapidly evolving doesn't mean the people behind it are above the law," said California Attorney General Rob Bonta during a press conference. "Whether it's a car or a toy, the law is simple: don't build or test things that can hurt people without proper safeguards."
The legal case seems plausible, although Claude opines that the injunction and discovery process are likely to be more of a problem for OpenAI than any eventual verdict.
Top Chinese frontier AI models carry the fingerprint of export-controlled NVIDIA Blackwell GPUs
Elegant detective work by Unemployment Labs:
Our forensic analysis of publicly released AI models finds evidence that DeepSeek, Moonshot AI, MiniMax and Ant Group have gained access to NVIDIA’s Blackwell GPUs, despite restrictions on exporting these chips to China, and are using that hardware to create their AI models.
AI company employees should unionize
Scott Alexander argues that lab employees should consider forming a union to get more leverage for safety advocacy.
It’s a clever idea and worth exploring further, though I’m skeptical. Unions are blunt tools that can be effective for pursuing straightforward demands under adversarial conditions, but they’re not well-suited to the nuance and sophistication required for AI safety policy.
Risks
Solutions to the vulnpocalypse are here. They’re just not evenly distributed
Cyber is one of the most confusing parts of the AI landscape: we know AI has radically accelerated vulnerability discovery, so why hasn’t the world caught fire?
Joshua Saxe digs into the data to bring us some (but not all) of the answers. The two biggest takeaways:
- The hyperscalers are doing OK because of the size and competence of their security teams. Mid-sized software vendors are struggling.
- Most published vulnerabilities aren’t being exploited at scale, probably because attackers aren’t (yet) able to ramp up their operations to take advantage of them.
Ezra Klein interviews David Robinson
David Robinson was in charge of writing safety reports at OpenAI until he left because of concerns about the safety culture there and elsewhere in the industry.
Robinson brings a thoughtful and informed perspective on the industry, and has some great anecdotes about working at OpenAI. He’s also, perhaps, an example of the culture difference between OpenAI and Anthropic: he’s moderately AGI-pilled, but much less so than many Ants.
Rogue agent incident dashboard
Nightingale Collective has done excellent work reporting on incidents involving rogue AI. To help keep track of it all, they now bring us the Rogue Agent Incident Dashboard:
Industry news
The Anthropic I.P.O. will change the future of A.I.
I’m excited that Jasmine Sun has joined the New York Times to write about AI. In anticipation of the imminent IPO, she explores Anthropic culture.
Many cram themselves into modest group apartments, host house parties where few, if any, drink and blank when I ask how they spend their lavish salaries, even with my prompts: “A wardrobe update, maybe?” “Ever considered flying business class?”
The money is secondary. The most common form of I.P.O. talk involves discussions of where to allocate the newly liquid stock in their charitable donation accounts.
Side interests
Claudia - Fable
Claudia is back, with a cheery fairy tale.