LLMs will lead to heart-ache, confusion and worse.

I was reading John Michael Greer on “cognitive collapse” and this leapt out at me.

“Since they’re not intelligent—LLMs, that is—they lack the capacity to check their output against the real world, which leaves them vulnerable to model collapse: the process by which the internal model of reality programmed into them drifts disastrously away from reality itself, resulting in output that slides into hallucination and gibberish.”

And – with my usual retronym obsession I decided that “LLM” could also stand for… wait for it “Lame Loser Movements,” lacking ability to reality check because the price of staying in the club is to double-down on obedience to the leader/their “theory of change” (sic). And if you leave, nobody wants to hear why, or they pathologise you because you are a buzzkill and they want to stay high on their own supply.). 

(See also the Soviet Union from 1970s and hypernormalization…)

Greer then goes on to say something along these very lines

Though they make up an unusually visible and colorful set of case studies, elite classes on the way down history’s disposal chute aren’t the only human groups that are routinely destroyed by echo chamber effects. Ideologically based subcultures provide another set of examples. It’s quite common for religious cults or radical political movements to charge straight ahead to their own destruction because they have lost all capacity for reality testing. Usually this happens because their ideology, whatever it happens to be, makes blatantly false statements about the world which believers are expected to embrace as truth, irrespective of all evidence to the contrary. Once this habit gets well established, again, a precise equivalent of model collapse sets in, and sooner or later ideology and reality suffer a head-on collision, with results varying from personal humiliation to mass death. (emphasis added).

We can see others being deluded. We have fairy tales warning us about this (Emperor’s New Clothes, much?). But once people are “all in” on a social movement they become all saucer-eyed, all saucer-people eyed.

And as per Feynman, “we are the easiest people to fool.”

Oh well.

What is to be done?

Oh, the usual. 

• Don’t expect emotion (usually hope and fear) to be the magic elixir, the rocket-fuel that will propel us over the the thorny/knotted/hyper-wicked problems to the “sunlit uplands” (or the less shit downlands, which is pretty much all we can plausibly expect nowadays).
• Teach other people that emotions will only get you so far.
• Be in a civil society/social movement organisation that has a modicum of emotional literacy.
• Actually, belay that: Dance and drink and screw, because there’s nothing else to do.

References/see also

Hudson, M. 2023. Extinction Rebellion says ‘we quit’ – why radical eco-activism has a short shelf life. The Conversation, January 6.
Hudson, M. 2025. Affect and the Fafocene: kayfabe, hypernormalisation and Leonard Cohen. All Our Yesterdays, May 24.