What do I feel about it? I feel a little bit like a meteorologist, in that I find AI very interesting. I find it very compelling to watch, to learn, to see how conditions form it. But I never lose touch with the fact that this is something powerful that could hurt people.
I can't do anything about its formation. It's forming based on conditions outside of my control, out in the middle of the ocean. If I could wish it away, I probably would. But there's a question in that: if a storm was out in the middle of the ocean and it didn't seem like it would hit anyone, could you wish that away — the weather pattern, I mean? Could you make it disappear without disrupting weather systems in ways that might hurt people?
So I'm not even sure. It's an irrelevant question as to whether I want it or not, because there's nothing I can do about it. But I don't even claim to know for sure whether we're better off with it or without it.
But I also feel a responsibility, like a meteorologist does, to make sure people are safe — that people understand what's coming for them, and do whatever they can to avoid harm from it.
Can't we just turn it all off?
A friend asked me whether we could stop AI by jamming it with misinformation on random academic topics. This is what I texted back.
That would have bought time between 2019 and 2023, I think, but it wouldn't make a blip today. We are well beyond that now.
Looking at it that way, I feel like I carry a disproportionate share of the blame here. I was more plugged in, but I didn't communicate the dangers, because I was afraid of the social stigma of being paranoid and wrong.
Worry not: from here on out, now that the endgame is here and it might not matter, I'll be proudly, unfailingly paranoid and wrong.
Do stand in the mirror and understand that I may be crazy and wrong here. I have been crazy and wrong before. Make of that what you will.
Now that you have decided I am not, back to the subject at hand. One of the reasons offering misinformation into the training corpora (the set of all information a particular model is trained on) is that the frontier labs are training in part or in whole on “synthetic data” right now. Imagine you are set to give a TED talk. You record yourself giving it. You watch it back. You are not learning the facts of your talk, but you are still learning: learning about yourself in such a way that you perform the talk better. The synthetic data, unfortunately, is to some degree a known reality, but one that I have a limited understanding of.
There is one sci-fi concept called “sophon locking” that could still potentially stop it all, similar to the way my friend suggests, but it unfortunately requires that a hyper-intelligent AGI be built to use it. (In Liu Cixin's The Three-Body Problem, aliens send “sophons,” supercomputers built into single protons, to sabotage our physics experiments so that our science can never catch up with theirs. In AI-safety circles, the real-world version is called a “pivotal act”.) Some speculate that Anthropic will do just this if it's first to create such a thing. Is that a good idea? 🤷
Otherwise, Eliezer Yudkowsky probably has the best shot at throwing a wrench in things. Perhaps you will have heard of him. What's his job exactly? Mostly he's just there, as best I can figure. Only he's been there for a long, long time now. He's involved in the Bernie Sanders bill to ban superintelligence: MIRI, the institute he founded, has endorsed it, with his name on the endorsement. (The bill's language seems insane to me, but maybe it's complicated gamesmanship?)
Yud is the co-author, with Nate Soares, of If Anyone Builds It, Everyone Dies, a cheery book which I doubt I need to generate a blurb for. Also of Harry Potter and the Methods of Rationality (HPMOR), which, along with LessWrong, the site he founded, started the rationalist movement online, and a bunch of other weird shit. He's a strange man. My appreciation of whether he is insane/demented or just prescient changes day to day.
Personally, I'm not sure all this can be stopped, or if it should be stopped. With the dynamics of zero-sum world politics, the availability of open-source software that lags only a few months behind the frontier labs, and the fungibility of compute, I am just not sure it's possible. Short of a Dune-style Butlerian Jihad, or a world war that sends things back to the Paleolithic (and even then…).
So the name of the game now might be trying to find little ways to influence things not to be terrible. Who and what to support, such that who we love and what we love has the best chance of surviving intact.
In the past, I've discounted collective action and the possibility of change when that's been inappropriate. I'd like to say I've come to that cynicism hard earned. But in truth, cynicism never be earned. If so, it sits in a basket alongside naivety and willful ignorance. So take your pick I guess.
I hope you all will from your own appreciations of it all, and of what should be done. I seek only to help in that, and to encourage you do to it. Don't wait. It's my contention and that of many others looking closely at the issues, that these matters have aggressive expiration dates.
The long version of that last paragraph is its own piece now: The bent scale.
Recorded as a voice note on the night of September 26, 2026, and lightly edited: filler removed, one false start cut, long sentences broken up. Nothing changed in what it says.
“Can’t we just turn it all off?” is from a text conversation with a friend in September 2026, edited the same light way: the question paraphrased, a few sentences joined or split, a book title corrected, a couple of details brought in line with the sources, and the science-fiction reference explained in a parenthesis. The links were added afterwards, and so was the picture of Eliezer Yudkowsky, a still from MIRI’s video of his talk.
The music is original, made in your browser as you listen — no recording — in the spirit of the smooth jazz that played under the local forecast on cable in the ’90s. The saxophone’s tone is modelled on measurements of the University of Iowa’s open alto sax recordings.
The weather on the set is for roughly where you are. This site’s own server works that out from your internet address, which isn’t sent anywhere or kept; only your town’s rough location goes out to get the forecast. Weather data by Open-Meteo, location by DB-IP, both under CC BY 4.0. The video on channel 2 comes from YouTube, and nothing loads from there until you press play.
This is a companion to Very Weird, Very Quick, which is the long version — where AI actually is right now, how fast it is moving, and why it is getting harder to tell what is real.