Summoning the Demon is the Easy Part
Yours isn’t the only agenda
AI politics will be about politics, not AI
- Anton Leicht
Epistemic status: thinking out loud.
Inkhaven status: There’s good stuff here, but it needs further revision. But Ben Pace is a cruel taskmaster, and I must publish one last time or else perish forever.
Friends, let us be honest with ourselves. Very few of us ended up in AI safety because we’re good at politics. We’re good at many things: AI safety is full of alarmingly talented people who just happen to be famous authors, or performing musicians, or chess prodigies.
But politics? The dark art of scanning a crowd and knowing what they’re most afraid of? Intuitively knowing what dream to spin for someone to convince them to follow you to the ends of the earth? Very few of us have those gifts.
To our collective credit, the AI safety movement has realized that politics is going to be critical to our success. Beards have been neatly trimmed, dress shirts purchased, and talking points polished. For the greater good, we are learning how to summon the demons of politics. And the demons have answered our summons: AI is quickly becoming salient and politicians are starting to introduce AI legislation. Maybe this isn’t so hard after all?
We are about to get a rude awakening.
Summoning demons isn’t as hard as they say
Imagine that through careful planning and networking, you’ve managed to arrange a meeting with Senator McBigly. You’ve done your homework and you’ve decided that the best way to reach them is through their interest in child safety.
So today’s talking points are about children. You’ve got charts about what social media has done to the mental health of teenage girls, and transcripts of an LLM coaching a teenager on how to kill himself. You’ve rehearsed a beautiful speech about how tech companies—and especially AI companies—are reckless and inhumane. They can’t be trusted to protect our children, and they can’t be trusted not to kill humanity.
The meeting goes great. McBigly listens to your presentation and asks good questions, and at the end of the meeting they agree that Big AI must be brought to heel. On your way home, you congratulate yourself for adding an important senator to your army of dedicated AI safety advocates.
I just have one question. Senator McBigly has a fancy corner office and a long list of committee positions because for the last thirty years, they have mastered the brutal art of winning at politics. Do you suppose they are congratulating themselves on joining your army, or do they perhaps have a different understanding of what just happened?
In Soviet Russia, demon summons you
Let’s revisit how the meeting went down.
Your agenda is AI safety—you’ve come to Congress because you want to build a coalition that will advance that cause. You are (I hope) sincere when you tell McBigly how much you care about child safety, but ultimately what you care about is legislation that reduces existential risk.
McBigly, knowing at least as much about politics as you do, is playing the same game. They are (possibly) sincere when they agree with you that existential risk is bad, but AI safety isn’t their focus. They are adding you to their coalition because they believe you can be useful in advancing whatever their agenda actually is—and when push comes to shove, that agenda is what they will prioritize.
You can summon a demon for any reason you like, but the demon comes for its own reasons, not yours.
Toward a taxonomy of demons
If you’re going to summon demons, you should probably understand their motivations. For today’s purposes, I submit that there are three broad classes of political demons: the Ideologist, the Power Seeker, and The People. (Probably most politicians are a mix of Ideologist and Power Seeker—but at any given moment, they will behave as one or the other).
The Ideologist
Bernie Sanders is an Ideologist. He’s dedicated his career to fighting for the workers, against the billionaires. That’s the lens through which he sees the world, and it is what drives his decisions.
To be clear: Sanders understands that AI presents an existential risk, and he is sincere when he talks about the importance of addressing that risk. But for him, AI safety is battle #347 in the war of the workers against the billionaires. That’s very clear when you listen to him speak, and it’s very clear if you read the text of his data center moratorium.
The Power Seeker
Ron DeSantis is a Power Seeker. He pursues power for its own sake—he may be sincere when he talks about a particular issue, but the issue isn’t the point. The point is that he can leverage the issue to build his own power. Power Seekers typically have a preferred way of securing power—DeSantis has found success provoking and harnessing populist grievance against the elite.
DeSantis might well be sincere when he talks about the dangers of AI. But at the end of the day, he rails against AI not because he’s deeply committed to that issue, but because it’s a useful tool for generating the populist anger that keeps him in office.
The People
We are a democracy—all legitimate power in this country derives from the people and must serve the people. But we are a representative democracy rather than a direct democracy for a reason.
Most voters are busy with their own lives and priorities, and it isn’t reasonable to expect them to have a deep and nuanced understanding of every issue. They know whether they think climate change is an urgent priority or a globalist hoax, but they don’t know how many ppm of CO2 are in the atmosphere, or what the Atlantic Meridional Overturning Circulation is. Nor should they be expected to.
What this means for AI safety is that the average voter will know whether they are pro- or anti-AI, and they will support or oppose legislation based on whether they perceive it as pro- or anti-AI. But they will not engage deeply with the specifics of any particular AI-related policy.
The politics of AI safety are cursed
So far, so good—this is just how politics works. You find a suitable politician, figure out how to frame your agenda in a way that approximately aligns with theirs, and pray that whatever emerges at the end of the legislative cycle is more or less what you wanted. People summon demons all the time and mostly don’t get eaten.
Unfortunately, I don’t think that game plan is going to work for us. The politics of AI safety are cursed in ways that make us useful allies for many politicians, without making them good allies to us. Two factors work against us:
- Our agenda is expensive
- Our agenda is almost but not quite the same as many others
Our agenda is expensive
An AI pause would have immense costs. About 30% of the US stock market is highly AI-leveraged: a pause would crater the stock market and quite possibly trigger a recession.
No politician wants to be known for causing a recession, and the voting public isn’t keen on the idea either.
Our agenda is almost but not quite the same as many others
To a casual observer, the AI safety agenda seems to overlap with many others:
- Because we are somewhat in conflict with the big labs, our agenda is almost an anti-tech agenda.
- Because there is a lot of money in AI, our agenda is almost an anti-capitalist agenda.
- Because there are concerns about AI’s impact on children, our agenda is almost a culture war agenda.
And yet, none of those are actually our agenda. AI safety is not advanced by taxing the big labs, or requiring the data centers be built with union labor, or making sure that children aren’t exposed to woke AI.
Asymmetrically useful
As they say in poker, “If you’ve been in the game 30 minutes and you don’t know who the patsy is, you’re the patsy.”
- William Buffett
We are useful to many others
When we campaign for AI safety, we strengthen a broad anti-AI sentiment among the public. That sentiment is broadly useful to a wide range of politicians:
- Populists are happy to capitalize on it boost their anti-elite positioning.
- Socialists are happy to leverage it to drive wealth redistribution.
- Culture warriors are happy to use it as a pretext to punish big tech.
But they are not useful to us
And yet, none of that helps advance AI safety. It’s all anti-AI, but it does very little to actually reduce existential risk.