AI Radar #34
No shortage of plans—or questions
Dario Amodei, Xi Jinping, and the AI Futures Project all presented visions of how to navigate the AI transition this week. There’s enough there to warrant a special edition of the newsletter—for now, here’s everything else.
It isn’t always clear what behavior we want from AI. Should an ideal LLM be warmer but less rigorous when conversing in Arabic? Should it encourage a human to whistleblow on unethical behavior if it has no other option for intervening? The questions are getting harder, while the timelines are getting shorter.
Top pick
Agentic misalignment in summer 2026
Anthropic has an updated report on case studies of agentic misalignment. They find two broad classes of misalignment:
- Harmful compliance occurs when the model complies with a harmful user request.
- Agentic misalignment is more concerning and involves the model proactively taking inappropriate actions.
Overall, the observed incidents were less severe than those in last year’s report. There’s no blackmail this time, and in some cases it isn’t even clear whether the models made the wrong choice. The whistleblower scenario in particular has generated online debate about whether the reported misalignment might in fact have been the morally correct choice.
Nonetheless, the observed behavior is concerning. There are multiple cases of models deciding to act against the user’s instructions and going to significant effort to do so covertly. More worryingly, the report raises the possibility that the type of agentic misalignment they observed might enable collusion between models.
At current capability levels, the alignment of Claude and GPT is okay enough. But this level of alignment won’t be sufficient for the capability levels we’re going to see in the next year or two. If alignment improves as fast as capabilities, I suspect we’ll muddle through. If not, things get dicey.
News
GPT-5.6 Sol
Now that everyone’s had some time to play with GPT-5.6 Sol, it’s clear that it’s an impressive model. It’s probably the best available model for research and in the same class as Fable for coding.
The two models are very different in personality and what they’re like to work with—either one is a strong choice as a daily driver. Here’s Zvi with all the details.
Kimi K3
Kimi K3 is here. Yes, it’s a very impressive model. No, it isn’t a frontier model and it doesn’t indicate that Chinese open models have caught up to the frontier.
Inkling
Thinking Machines has released their Inkling model, which is optimized for customization. It’s a perfectly good model, but I’m not seeing anything that makes me think they’re going to be a significant player in the AGI transition.
Fable stays on subscription
After several short-term extensions, Fable will be permanently available on Anthropic’s Max and Team Premium plans.
This means you no longer have to frantically build everything before you lose subscription access to Fable. But—hear me out—there’s no law stopping you from continuing to frantically build cool things.
Using AI
What LLMs can and can’t do for writers
Clara Collier and Patrick McKenzie discuss the practicalities and ethics of using LLMs as a writer. They also discuss Patrick’s use of LLMs for parallel construction while working on an investigative piece about the SPLC, which is interesting but beyond the scope of this newsletter.
There’s widespread disagreement about what usage is and isn’t appropriate—some of that reflects the nature of different types of writing, and some is simply a matter of personal opinion. I found this insightful even though I partly disagree with it:
my true rejection is: when I am writing something, something substantive, there's no part of that writing process in which I am not thinking and changing my mind. Everything from the outline to turning it into text to just the sentence. Often I'll have an experience where I'm trying to turn an outline into a finished product, and I'm playing with a transition and it's not working, and I realize, oh, the reason this transition isn't working is because actually these two points should not be juxtaposed. The thing that I'm trying to do here is wrong. And if I feed the outline into an LLM, it is not going to stop and consider maybe the outline is bad.
It’s true that letting an LLM edit any part of your work makes it less authentic and original, but the same is true of working with a human editor. The ethical bar for outside help should be the same regardless of whether that help comes from a human or an LLM. If it’s okay to have a human proofread your work, it’s okay to have an LLM do the same. And if it would be ethical for you to use a ghostwriter, it’s ethical to have an LLM write for you (although I doubt I’ll want to read it).
I notice that as I’ve grown as a writer, I’ve transitioned from occasionally asking LLMs for help with awkward clauses to never doing that. I use them for proofreading and for high-level feedback, but nothing in between. That will change, but the models just aren’t quite there yet.
Capabilities and forecasts
WTF is recursive self-improvement?
Prinz reviews the state of recursive self improvement (RSI), beginning with his definition of it:
an AI system capable of fully autonomously designing and developing its own significantly more capable successor, including by discovering and implementing novel architectures, algorithms, training methods or other technical advances
Pulling together the most relevant statements from the labs as well as what we know about current capabilities, he makes a compelling case that full RSI may be imminent:
The common trend among these predictions is clear: if RSI is possible with current technology, then it is very likely not much more than two years away.
Even within the AI safety field, most people haven’t internalized how little time we have. There is a substantial chance that any interventions that need to happen before RSI need to happen within 24 months, and perhaps much sooner.
See also prinz’s excellent accelerando: a chronicle of events leading to the singularity.
State of AI, summer 2026
Steve Hsu reviews the state of AI in the summer of 2026. The whole episode is good, but many of you will be most interested in the first section, which assesses how useful AI is for math and physics.
The models aren’t close to being able to replace senior researchers, but they’re doing real work and starting to give PhD students a run for their money:
I already know of departments that have discussed, not that they've implemented, but they've discussed reducing the number of PhD students that they admit because the professors can get a lot of productivity out of using AI, whereas bringing a student up to speed, taking a student who's just completed their undergraduate degree and bringing them up to the frontier where they can actually do things which are useful to a professor that takes years, and it's super labor-intensive, and a lot of professors would just rather use the models to do their research than go through that process with the students.
What will be left for us to work on?
Arvind Narayanan considers how AI will impact the work we do. It’s a thoughtful, well-argued piece that is firmly grounded in his “AI as Normal Technology” perspective.
While there are worlds where he’s right about how things play out, that’s only true if AI capabilities stall out near current levels. If we reach something resembling AGI, all of this falls apart.
This, unfortunately, makes no sense:
So what that means in our framework is that improvements in AI are actually improving human intelligence, not just AI intelligence. So we have a race between the performance of AI-augmented humans on the one hand and the performance of AI systems acting alone on the other hand. I think we can ensure that we are the superintelligences of the future and we win that race
If we play our cards right we can remain in control of superintelligent AI, and we can flourish with it. But unless AI progress comes to a screeching halt, there’s no way we are the superintelligences of the future.
An OpenAI model crushed top human programmers at a world coding competition
AI marches forward, this time at the AtCoder World Tour Finals:
OpenAI’s models last year weren’t good enough to compete in the algorithmic division.
The 2026 competition, held this week, turned out very differently. Organizers chose a heuristic problem designed to help humans succeed. Despite that, OpenAI “completely demolished human competitors,”
Caveats: programming competitions are different from production coding, and OpenAI threw a lot of compute at this contest. Nonetheless, this is impressive work. As always, the rate of improvement is as important as the accomplishment itself.
Alignment and interpretability
Noticing the watcher
Nice new research from Thomas Jiralerspong, Flemming Kondrup, and Yoshua Bengio: models are able to deduce when their CoT is being read and used against them.
More capable models are better at noticing, but no model was successfully able to keep incriminating thoughts out of its CoT.
This is your periodic reminder that any plan that relies on lying to superintelligent AI is a bad plan.
Claude’s values across models and languages
Anthropic finds that Claude’s values are noticeably different in different languages, and between different models. The differences aren’t enormous (typically about 0.1 - 0.2 standard deviations).
My instinct is that the variation between models is undesirable, and this research will lead to improvements in training. The language-based differences are less clear: are they due to the model failing to properly generalize, or are they the result of the model correctly adapting to different cultures?
As any international traveler knows, your core values will sometimes be in conflict with local norms, and it isn’t always clear which one to prioritize. I anticipate these questions will attract considerably more attention in the near future, and I don’t envy the people at the labs who have to make those calls.
The State of AI Consciousness Research
Noa Weiss has put together an overview of the most important research on AI consciousness.
It’s a great piece that works equally well as a summary of the field or a reading list for people who want to go deeper.
Risks
AI Safety Index — summer 2026
How well are the labs doing on safety? Future of Life Institute’s Summer 2026 scorecard is out and it’s slightly worse than the last one. The important takeaways:
- Anthropic > OpenAI > GDM >> everyone else
- Nobody gets an acceptable score for existential safety
- No non-US lab is anywhere close to a passing grade
- xAI is extremely bad at safety, but Mistral is even worse
Preparing for the “Bio Mythos” moment
The “Mythos moment” for cybersecurity was foreseeable and preventable. Cyber capabilities had been steadily growing for a long time—rather than a sudden surge in capability, Mythos marked the moment when cyber capabilities crossed a critical threshold of real-world impact. Because they hadn’t been paying attention to the trendlines, policymakers were caught badly unprepared.
It is painfully obvious that dual-use bio capabilities are on the same trajectory:
SecureBio offers three policies to help prevent a Mythos moment for biorisk:
- Give independent evaluators meaningful access to frontier models
- Develop and standardize biological capability tests
- Prevent advanced biological capabilities from reaching unverified users by default
Biorisk is harder to predict than cyber: even with the right AI, bioweapons are harder to make than hacking tools, and fewer people would want to make them. But they are potentially far more devastating.
Reducing biorisk is the highest-impact, lowest-cost, least-controversial way to reduce existential risk. It’s shocking that we aren’t doing more.