Against Moloch #45
AI video killed the radio star
We live in an age of glorious abundance, when anyone with a half-baked idea and a Claude subscription can produce a four minute music video with charming animations and a catchy refrain.
The day will come—probably by Thursday—when we’re over it. We will realize that every new video feels strangely similar to every video that came before it, and we’ll roll our eyes and move on to the next thing.
But this is not that day. Today, we revel in the bounty of the AI age.
Top pick
The Dot and the Swarm
Ethan Mollick learns the Bitter Lesson:
In the last year I have been posting about how I suspected that humans would have to approach working with agents as a manager, deciding how to delegate work to agents and specifying how those agents should be organized. I thought that getting agents to work effectively as a group would take careful construction, akin to building a company, and that this would take time to figure out.
Nope.
[…] It turns out that organizing work is just one more thing AI can learn to do.
A common argument for slow timelines is that it’ll take years for firms to figure out how to diffuse AI into their existing processes. That misses the point: when AI gets good enough, the solution to the diffusion problem is simply to stay out of the way while the AI diffuses itself.
The Bitter Lesson is easy to forget, even in an article about it:
But much of management exists to solve problems agents don’t have, and agents increasingly work through the same messy systems people do, even on ambiguous tasks. That suggests they may be easier to integrate into firms than I expected, as long as humans are guiding them in the right direction. [emphasis added]
I respectfully suggest that this otherwise excellent article will age better if the bold text is removed.
The included video about the Bitter Lesson (which Claude one-shotted) is outstanding—this was the first AI video to really punch me in the gut.
The view from inside
This is so good I couldn’t resist having two top picks.
Palisade Research brings us frominside.ai, a collection of video interviews with current and former lab employees talking about existential risk.
These are some of the smartest, best-informed people in the field: I highly recommend watching at least one or two.
Industry news
Yes, model releases are getting faster
between anthropic and openai, a new model used to come out every ~10 weeks
now it's every ~11 days
DevDay 2026 Recap
OpenAI just held DevDay 2026—here are the most interesting announcements:
Dots are the most interesting new feature:
Dots are remarkably capable, always-on agents built to handle everything. They’re a whole new way to work with AI—one that gets to know what matters to you, is always working on your behalf, and takes important work off your plate so you get more of your time and attention back.
Always-on, persistent agents have been around for a while, but this looks like a big step forward for making them easy enough for mortals to use.
ChatGPT Space is a collaborative work environment. AI-centric collaboration has been super awkward—it’s great to see both labs rolling out features to improve that.
Sign in with ChatGPT lets you use your ChatGPT plan to power AI features on participating sites. This is a useful solution to a real problem, and also a clear attempt to lock people in to OpenAI’s platform.
Ultrafast lets you run Astra up to 8x faster than normal. Painfully expensive, but potentially worth it. I’m planning on trying it out next week, and am afraid I may conclude it’s worth the substantial additional cost.
Introducing GPT-6.1 Sol
Remember how last week’s newsletter covered the launch of GPT-6 Sol? Here’s GPT-6.1 Sol. Super capable, very reasonably priced, probably not gonna be your daily driver.
Introducing Gemini 4 Argon
Google has announced but not released Gemini 4 Argon. It benchmarks better than I expected: this is a credible new release.
We’ll see how it looks after people have a chance to play with it for a few days, but I expect it’ll be like the best open models: very good, expecially on benchmarks, but not in the same league as the frontier models.
Claude Sonnet 5.5
This looks like an excellent model whose performance is close to Opus 5.5 in many but not all domains. Nonetheless, most people should probably just use Opus (or Fable, or Astra, in that order).
America.gov
Nice—the federal government has a new chatbot for unified access to government services. First impressions: it’s not great compared to frontier AI, but it’s probably the best way to find many government services online.
Capabilities and forecasts
Waymos are safer
Waymo is commendably transparent about their safety data—possibly because they’re a virtuous company, and possibly because their safety record is worth bragging about.
Per mile driven, Waymos have 95% fewer crashes with serious injuries and 93% fewer pedestrian crashes with injuries. The time is coming when operating a car manually will seem shockingly negligent.
How are AI-2027’s forecasts holding up?
Daniel Reeves checks in on how AI-2027’s forecasts are holding up. Predictions are hard, but these have held up quite well:
Alignment and interpretability
Inside Anthropic’s quest to instill morality into its AI models
The New York Times has noticeably leveled up its AI reporting this summer. Here they bring us an in-depth report on Anthropic’s efforts, led by Chris Olah, to cultivate dialog with religious leaders:
Quietly, Mr. Olah and his team at Anthropic have been on a frantic and high stakes quest to understand the A.I. models they have unleashed on the world and to convince the models to abide by a moral code that will protect humanity.
To do that, Mr. Olah began reaching out last fall to religious thinkers from different faiths. What followed was an extensive series of confidential meetings, calls and discussions
Strategy and politics
The White House Accord on [Artificial Intelligence]
The White House just hosted a gathering of AI leaders, which produced a “morally binding” agreement on AI safety. It’s significantly better than nothing, while still falling far short of what we need.
Victoria Krakovna brings us the text of the agreement, and Zvi brings us a detailed report:
When you have one side that cares mostly about vibes, and the other that cares about the substance, this suggests a deal that can be struck.
Suddenly everyone agrees on everything. Works for me.
I fully endorse Zvi’s observation:
The other supposed agreement was a distinct executive order to change the name of AI to [AI], which Trump claims the lab leaders signed off on. I’m sorry I have this tick, I try to type [artificial intelligence] and instead I end up with brackets.
The Preference Cascade Is Only Getting Started
Jacob Coxon’s resignation from Anthropic was the catalyst for a profound shift in how seriously the American public takes AI risk. It’s too early to tell where the political battle lines will be drawn, but policy options that seemed impossible only a few weeks ago are now very much on the table.
Zvi reports on the early stages of the AI preference cascade.
Ezra Klein asks why we are sprinting off the AI cliff
Ezra Klein is on a roll, with two excellent (but not perfect) videos this week.
Why are we sprinting off the AI cliff? is an excellent presentation of the key issues, with a nice explanation of the hugging face incident. He ends with a beautiful reference to the tragedy of Circe:
This is the tragedy of[OpenAI and Anthropic’s] work: fighting against the prophecy, they have drawn it tighter around their necks—and ours.
I have only one quarrel with this otherwise excellent video. Klein raises the coordination problem at the heart of the AI race, but doesn’t fully engage with it. His proposed solution to the race is to force the US labs to stop, without doing anything about the far more reckless Chinese labs six months behind them. Which, unfortunately, only makes the situation worse.
Without a solution to the full problem—not just the easy part—Klein’s fight against the prophecy would simply tighten it further around our necks.
Ezra Klein talks to Bill Gates about AI
Another great Klein video, this time with Bill Gates. Plenty of peole have pointed out why Jensen Huang’s approach to AI safety makes no sense, but it’s great to see Bill unleash his trademark fury on it.
Bill is insightful, forceful, and well-informed, although strangely not ASI-pilled: he sees clearly why we need to freak out about misuse risks, but doesn’t seem to understand why loss of control is also an imminent crisis.
The Ban Artificial Superintelligence Act
Bernie Sanders and Greg Casar have introducted the Ban Artificial Superintelligence Act:
legislation to stop artificial intelligence (AI) oligarchs from building machines humans cannot control and to pause advanced AI development until we have put in place rigorous federal testing and oversight to ensure AI does not harm humanity.
Like Sanders’ previous legislative efforts on AI, this is simply a very bad bill. It is interally contradictory, has significant constitutional problems, and would cause massive collateral damage. This isn’t a bill to ban the development of superintelligence—it’s a bill to immediately ban all modern AI, in the most chaotic way possible.
I’m not going to go through it section by section, but I want to call out one representative issue. Section 9 prohibits the development or deployment of a system that displays any “superintelligence precursor characteristics”, including:
The capacity to access secured digital or physical infrastructure, such as a computer information system or network, without authorization or in excess of authorized access.
So any level of offensive cyber capabilities counts as a “superintelligence precursor”. Which is nonsense on the face of it, and means that the bill would immediately ban every significant model that’s been released within at least the last year.
I won’t speculate about whether the overreach is deliberate or simply the result of incompetence. Either way, this bill would be a disaster. AI safety desperately needs strong federal legislation, but this isn’t it.
Rob Ennals has a more detailed analysis of the bill, and I’m disappointed to see that MIRI endorses it.
Pacing the frontier: an agenda
This is an impressively comprehensive (and, at 18,000 words, brutally long) overview of the complexity involved in developing an effective pacing strategy.
Pacing effectively is more complicated than it sounds, with numerous non-obvious but important details. There are plenty of good solutions here, but this is probably most valuable as a list of problems that will need solutions.
Inaugurating The Era Of Super Intelligence
Further comment would be superfluous:
To the maximum extent permitted by law, executive departments and agencies (agencies) shall use “Super Intelligence” and “SI” in place of “Artificial Intelligence” and “AI” in official correspondence, public communications, websites, reports, policy documents, and other non-statutory documents within the executive branch.
Risks
GLM-5.3 and the spread of advanced cyber capabilities
Why haven’t we seen the cyberpocalypse that some people predicted when Mythos was announced? In part because until now, models with advanced cyber capabilities had robust safeguards, and models without good safeguards didn’t have advanced cyber capabilities.
Anthropic evaluates GLM-5.3 and concludes that it’s the first open model whose cyber capabilities are genuinely dangerous. As expected, its safeguards are weak and easily removed:
None of this is unexpected—these capabilities are showing up right on schedule. It’s hard to say how much damage this model will cause: things might get ugly, but it’s entirely possible that (for now) defenders have enough of a head start to mostly contain the damage.
This report has inflamed the debate about whether to regulate open models. I don’t expect that debate to last forever: before long, open models will become so dangerous it’ll be obvious they have to be regulated. The open question is whether that happens before or after the first catastrophic incident.
NVIDIA Open Agent Safety
NVIDIA introduces their Open Agent Safety Platform, a reference design for a multilayered, hardware-based system for enforcing agent safety:
Agent safety requires independent security controls. The internet was not made secure by requiring that web developers promise to be good. It became safe because the browser stopped trusting the code in the web pages explicitly.
We need to build this trust layer for agents.
I’m not in a position to say whether this system might be a useful addition to the AI control toolbox, but they’re wildly overclaiming what it could accomplish, even in principle.
NVIDIA is making a rookie mistake here. The hard part isn’t figuring out where to put your control system—the hard part is figuring out how to design a control system that can detect and defeat a superintelligent adversary.
Control is useful and important, but it isn’t the solution. At the end of the day, you solve alignment or you fail.
Side interests
OMG! We've found other agents!
In these challenging times, I think we can all use a cheerful song about finding new friends.
AI is a normal technology?
So you think AI is a normal technology? Cute.
