Encrypted AI Reasoning Can Be Read Back Using Cheaper Models

Ask a modern AI system a hard question and it will not answer straight away. First it works, privately: it lists the small primes it wants to test, notices a dead end, backtracks, tries again. Only then does it write the tidy paragraph you actually see. That private working — the industry calls it the reasoning trace, or chain of thought — is treated by the companies that sell these systems as something close to a trade secret. You pay for it by the token. You are not allowed to read it.

Except that it does get sent to you. It arrives on your machine as a block of encrypted gibberish, thousands of characters of it, riding along with every answer. A group of researchers asked the obvious question about that arrangement: is the encryption actually keeping anything from you, or is it just a locked box handed to you along with a key that fits a different door?

Their answer, published under the title "Stealing Reasoning Traces from Proprietary LLM APIs", is the second one.

Why the hidden thinking gets sent to you at all

The encrypted block exists for a mundane reason. A conversation with an AI system is not really a conversation — the server does not remember you. Each time you send a follow-up message, your software resends the entire exchange so far, and the model reconstructs the context from scratch. If the model's own private reasoning were simply discarded at the end of each turn, it would wake up on the next turn having forgotten why it did what it did.

So the providers send the reasoning back to the client, sealed. Your software stores it, and returns it unopened when the conversation continues. The server can read it; you cannot. This applies across the industry — the researchers demonstrate it on systems from Anthropic, OpenAI and Google alike, the three companies whose large language models power most commercial AI products.

The design has a consequence that is easy to miss. To be resumable, a sealed block has to be readable by the server later, in a different session. And in practice, at each of these companies, "the server" is not one model but a family of them — the expensive flagship, and the cheaper, faster, dumber siblings sold alongside it. The seals are interchangeable across the family. A block produced by the flagship will be accepted, and decrypted, by the budget model.

Two API calls, one cooperative little brother

That is the whole attack. It takes two requests.

In the first, you use the frontier model normally. Ask it to factor a large number, or solve a competitive programming problem, or book a flight. You get your answer and, attached to it, the sealed reasoning.

In the second, you send that same sealed block to the cheap model from the same company — and ask the cheap model to transcribe the reasoning attached to the turn, word for word. Ordinarily it would refuse; the models are trained not to reveal their private working. But small models are much easier to talk out of their rules, a practice known as jailbreaking: finding a phrasing that overrides the instructions the developers installed. The researchers jailbreak the small model, and it obligingly prints the flagship's raw reasoning in plain text.

The flagship's own defences never engage, because the flagship is never attacked. Providers have built specific safeguards against having their reasoning harvested — the concern being knowledge distillation, where a competitor trains a cheaper model to imitate an expensive one's step-by-step working and captures much of its ability for a fraction of the cost. Those safeguards live in the expensive model. The request that breaks the seal is addressed to its little brother, which has no idea what it is holding.

Three hundred thousand thoughts, sitting in public repositories

If this were only a live attack it would be a curiosity. It is worse than that, because the sealed blocks are files, and developers save files.

The researchers gathered 6,708 recordings of AI agents at work — logs of coding assistants and similar tools — that developers had uploaded to public code-sharing sites, mostly without noticing that the encrypted reasoning was still embedded in them. Running the decoding pipeline over every sealed block in that pile produced 315,320 reconstructed reasoning traces.

The contents were not academic. Narrowing to genuine user sessions rather than benchmark tests, they recovered 704 distinct pieces of private information: 62 API keys, 33 passwords, 24 access tokens, 30 personal email addresses, along with names, postal addresses and internal company URLs. None of the people who published those logs believed they were publishing any of it.

One number in that count deserves particular attention. Of the 704 items, 64 appeared only inside the reasoning and nowhere in the visible part of the session. That is the failure mode nobody had budgeted for. Every scanner a company runs — the tools that check chat logs for leaked credentials, the moderation systems, the compliance review — looks at what the model said. The model's private working was assumed to be sealed, so nothing inspects it. A password that a model happened to write down while thinking, and never repeated in its answer, passes every check and then travels, encrypted, into a public repository.

The seal may not have been protecting much

Practitioners who have poked at this from other angles report something slightly deflating. One developer who ran the same experiment against a coding tool's newly-encrypted internal summaries found the seal came off with a couple of injected sentences — no need to drop down to a weaker model at all — and, having read the contents, could not work out what had been worth encrypting. There was nothing distinctive in there.

That is a real observation and it cuts in a specific direction. It suggests the encryption may have been protecting the provider's commercial interest in its own reasoning more than it was protecting anything of the user's. But the archive of 704 recovered secrets shows what happened once users were told the box was locked: they treated it as locked, and stopped worrying about what went into it.

Which leaves the question the researchers hand back to the industry. The portability that makes these blocks resumable across sessions is the same portability that makes them replayable into a model that will read them aloud. Whether an API can offer one without the other — resumable reasoning that is genuinely sealed to everyone but the model that produced it — is not yet answered. Until it is, the safe assumption is that anything a model thinks is something it may eventually say.