LLMs feel intelligent for the same reason psychics do, essay argues
The essay's author, who researches the use of language and diffusion models in software businesses and previously wrote a book called The Intelligence Illusion, says many people are convinced that chat-based language models are intelligent, even though nothing in how large language models (LLMs) work explains why that would be true. An LLM, the author writes, is a mathematical model of language tokens: given text, it produces a mathematically plausible response, not a reasoned one, and every AI researcher and vendor to date has said as much. Two explanations are offered for the mismatch: either the tech industry has stumbled onto the early stages of a wholly new kind of mind with no parallel in biology, or the illusion of intelligence sits in the user's head rather than in the model. The author places themselves in the second camp, and now believes there is even less reasoning happening inside these systems than they thought before, describing many proposed LLM use cases as bordering on fraudulent pseudoscience.
The author's explanation is that the intelligence illusion runs on the same mechanism as a psychic's con, known as cold reading. The idea was prompted partly by a blog post by Terence Eden on how often chatbot replies contain Forer statements, generic-sounding lines that feel personally accurate to whoever reads them. The essay describes the classic six-step psychic con: an audience that self-selects toward believers, a staged scene, a statement narrowed down to a likely demographic, a burst of questions testing whether the mark reacts, a loop of statistically generic but personally phrased guesses, and an ending in which the mark is convinced the psychic has uncanny powers. The mechanism underneath all of it is subjective validation, the cognitive bias by which people treat a statement as accurate simply because it feels personally relevant to them. The author lists the psychic's specific toolkit of "validation statements": Forer or Barnum statements (vague lines like "you tend to be hard on yourself" that nearly everyone accepts as true of themselves), the vanishing negative (phrasing a guess with a "not" so it can be read as correct either way), the rainbow ruse (assigning a mark both a trait and its opposite), statistical guesses, demographic guesses, unverifiable predictions, and shotgunning a rapid series of statements so the mark remembers only the one that landed. The author argues chatbots produce the same kind of output: confident, generic, and phrased so it feels tailored to the specific person reading it.
The author also stresses that susceptibility to this effect has nothing to do with intelligence, and quotes several remarks attributed to believers in the current AI wave, including lines such as "This is real. It's a bit worrying, but it's real" and "You need to keep your mind open to the possibilities. Once you do, you'll see that there's something to it." Someone raised to believe they have a high IQ, the author writes, is more likely to fall for the effect than someone raised to think less of their own intellectual capabilities, because the con exploits a quirk of the human mind that everyone shares, not a gap in reasoning ability.
Key facts
- The essay argues the impression that LLM chatbots are intelligent comes from the same mechanism as a psychic's cold reading, not from genuine reasoning in the model.
- Chatbots, in this view, produce "validation statements" in the Forer/Barnum style that sound personally specific but are statistically generic, exploiting the cognitive bias called subjective validation.
- The author traces part of the idea to a blog post by Terence Eden on how often chatbot replies contain Forer statements.
- Six named cold-reading tactics are described: Forer/Barnum statements, the vanishing negative, the rainbow ruse, statistical guesses, demographic guesses, unverifiable predictions, and shotgunning.
- The author says susceptibility to the effect is unrelated to intelligence: someone who believes they have a high IQ is more likely to fall for it than someone who does not.
Why it matters
The essay directly challenges the widespread habit of reading chatbot output as evidence of reasoning or understanding. It argues that the feeling of being personally understood by an LLM is produced the same way a psychic produces that feeling in a mark: through statements that are statistically generic but land as specific because the listener supplies the personal meaning. If that framing holds, a large share of anecdotal claims about what chatbots "get" about a given person or problem says more about the listener's own cognition than about the model.
Who it affects
Anyone who judges an LLM's competence, insight, or empathy from the feel of a chat transcript, and anyone building products, evaluations, or public claims on top of that impression. The essay singles out people in the current AI wave who describe chatbot interactions as uncanny or as evidence of something more than statistical pattern-matching happening inside the model.
How to use it
The essay is not a tool with a price or a signup; its practical use is as a checklist. When a chatbot's reply feels startlingly on point, the piece prompts the reader to ask whether the statement is actually generic and only feels specific because of subjective validation, and whether it matches one of the named patterns (a Forer/Barnum line, a vanishing negative, a rainbow ruse, a statistical or demographic guess, an unverifiable prediction, or shotgunning) before treating the exchange as evidence of understanding.
How solid is it
This is an opinion essay, not an empirical study, and the author says so directly, calling the cold-reading analogy "an idea I think explains the mechanism" rather than a tested finding. No dataset, experiment, or citation beyond the analogy itself and a nod to Terence Eden's blog post is offered to support it. The retrieved text carries no byline for the author within the article body.
Risks and caveats
The argument rests on an analogy to a well-documented con technique rather than on evidence that any specific chatbot is deliberately engineered to produce cold-reading-style statements, so it is a plausible interpretation rather than a demonstrated mechanism. The essay names no specific chatbot, vendor, or model as an example, and it gives no figures on how often or in what share of interactions the described effect actually occurs.
“This is real. It's a bit worrying, but it's real.”
— quoted in the essay as a typical remark from believers in the current AI wave