AI Reasoning Works Even When Its Written Steps Are Meaningless

In May 2026, a general-purpose reasoning model from OpenAI cracked the unit distance problem — a genuinely open question in mathematics, not a textbook exercise with a hidden answer key — and it did so on a single attempt. Months before that, machines of the same family took gold medals at the International Mathematical Olympiad, a competition so punishing that professional researchers list a medal on their CV for the rest of their careers. Google DeepMind, working with the mathematician Terence Tao, reported that AI had matched or bettered known results on 67 problems drawn from combinatorics, geometry, number theory and analysis.

Set against that: a steady drumbeat of research arguing the machinery does not work the way its vocabulary implies. A team at Apple described these systems as producing an illusion of thinking, with accuracy falling off a cliff on puzzles that were only modestly harder than ones they aced. Melanie Mitchell's group at the Santa Fe Institute found that models could ace benchmarks built specifically to test analogical reasoning — grids of visual puzzles — by leaning on shallow shortcuts rather than anything that generalises. As Quanta Magazine reported this week, the two bodies of evidence have been arriving more or less simultaneously, and neither side is obviously lying.

An index card's worth of certainty

Start with the vocabulary, because it is doing a lot of work. A large language model is a system trained to predict the next fragment of text. A large reasoning model, or LRM, is one of those trained further to first emit a stretch of intermediate text — a "chain of thought" — and feed it back to itself before committing to a final answer. The technique began in 2022 as a prompting trick: tell a chatbot to work through a problem in stages and its arithmetic improves. OpenAI's o1, released in 2024, automated the trick.

Asked what is actually established about all this, Mitchell offered three points compact enough to fit on a card. The models work: on tasks framed as reasoning, they beat their predecessors. The text they print while working is not necessarily a faithful account of what is happening inside them. And much of that text does no work at all — you can cut it out.

Strings of dots do the job as well as arguments

The second and third points are the strange ones. Subbarao Kambhampati, a researcher at Arizona State University and a former president of the main US artificial-intelligence professional body, has called these intermediate tokens "mumblings": language, certainly, but language whose meaning may be beside the point. His lab demonstrated in 2025 that swapping a model's correct working for wrong or irrelevant working left its performance on a formal task intact. The reverse also held. Trained exclusively on valid traces, a model would still sometimes print nonsense working and then produce the right answer anyway.

A 2024 paper from New York University pushed the point further, showing that literal strings of dots — filler with no semantic content whatsoever — could substitute for a human-readable chain of thought. William Merrill, one of its authors and now a professor at the Toyota Technological Institute at Chicago, put it bluntly: nothing guarantees that a chain of thought is "meaningful in any sense". Pavel Izmailov, an NYU researcher who has also worked on Anthropic's reasoning models, doubts that reinforcement learning — the standard training method — even pushes models towards honest traces. Maybe it does, he said, but he would not bet on it.

Delete half the steps, lose almost nothing

You might expect that even if the words are gibberish, the tokens themselves are load-bearing — the way a pinball machine runs on the metal of the coin, not the motto stamped on it. Researchers at Northeastern University and the University of California, Berkeley tested that on leading open-source reasoning models and found that somewhere between 30% and 60% of the "thinking steps" had almost no causal effect on the answer. Cut half of them and scores barely move. Weiyan Shi, one of the authors, warns against reading these traces as though they were wired to the output.

Kambhampati's group summed up the frustration in the title of a position paper at this year's International Conference on Machine Learning, which amounted to a plea to stop treating intermediate tokens as thoughts.

What the labs will and won't show

The people building these systems dispute the framing sharply. Sébastien Bubeck of OpenAI dismissed last summer's critical papers as science in heavy air quotes, saying the Apple findings stemmed from a training quirk in models since superseded, and that everything from GPT-5.5 onwards is unaffected. Apple did not put its researchers forward to respond. Asked whether the unit distance proof had been propped up by external tools — a formal theorem-checker such as Lean, say — Bubeck treated the question as beside the point. The chain of thought is public, he said; the model reasons the way a person does, and people don't use Lean.

What OpenAI published, though, was a rewritten summary of the model's working, produced by two human experts with the help of another OpenAI model. Since 2024 the company has not released raw traces, and Google DeepMind and Anthropic follow the same policy. So the document offered as evidence of how the machine thinks is, in the strict sense, a translation.

Retrieval wearing the costume of deduction

Kambhampati's own explanation starts where OpenAI's does — an LRM is a language model with more specialised training, no extra magic — and then diverges. He doubts a system trained to predict text would genuinely narrate its own procedure first, since describing a method is harder than guessing an answer. His hypothesis is "approximate retrieval": the model absorbs enough examples of what plausible working looks like to stitch together something that lands on a defensible result, without ever running a general procedure it could reapply reliably.

He is careful not to be mistaken for a denier. These are wondrous times, he says; his objection is to the field's appetite for flattering explanations, to claiming capabilities on the grounds that they will probably be true eventually. That, in his phrase, is not science. It is investment.

The practical stake sits underneath the semantics. Both Mitchell and Kambhampati note that state-of-the-art systems usually work inside ordinary software that checks and constrains them — the coding agents that have reshaped software engineering since late 2025, or DeepMind's proof system leaning on Lean. Where the scaffolding does the verifying, the trace need not be trusted. Where there is no scaffolding, the printed reasoning is all anyone has to inspect — and the research says it is a document that need not be true.