Latent Reasoning

Canonical version: Latent Reasoning.

Latent reasoning means letting a LLM think in vectors instead of words. A normal reasoning model writes its chain of thought as tokens: it picks a word, feeds it back in, picks the next one. A latent reasoner skips the "pick a word" step and feeds its internal state (a long list of numbers, the hidden vector) straight back into itself.

Why bother? Because picking a word throws information away. The hidden vector can hold several possible next steps at once; a token holds exactly one. So in theory, latent reasoning gets you the same answer with fewer steps, fewer tokens and less money. The best-known early work is Meta's Coconut ("Chain of Continuous Thought", December 2024).

The catch: you can't read it anymore

A written chain of thought has a nice side effect: humans (and other models) can read it. It's one of the few practical tools we have to see why a model did something, and to catch it when it plans something it shouldn't. In July 2025, researchers from OpenAI, Anthropic, Google DeepMind and others published a joint position paper calling chain-of-thought monitorability "a new and fragile opportunity for AI safety", and asking labs not to throw it away lightly.

Latent reasoning throws it away by design. That's why every latent reasoning paper now ends up in the same debate on Hacker News:

• "It's inspectable." Papers like Hybrid Reasoning Agents decode each latent step to its nearest token, so you get something to read
• "No, it isn't." The nearest token is a lossy summary of a vector that holds much more. You read a caption, not the thought
• "And training makes it worse." If you apply optimization pressure on the latent reasoning (e.g., penalize "bad" thoughts), you may teach the model to hide its intent instead of dropping it

My view: the efficiency gains are real, and someone will ship this. But a visible chain of thought is part of how I trust an agent that runs for hours on my machine. I'd want the agent's actions to stay readable at the very least, which is exactly the compromise hybrid approaches try to make.

Related ideas

• Looped transformers (e.g., Ouro) run the same layers several times on the same input. Several commenters argued that's just a deeper network with shared weights, and that latent reasoning is a close cousin
• Hidden thinking at frontier labs. Some labs already hide or summarize their models' thinking. Commenters guessed (from the size of encrypted thinking blobs) that it's still text underneath, but nobody outside knows for sure
• System 1 and System 2 Thinking. Latent reasoning is an attempt to give models fast, wordless thinking. Humans do most of their thinking that way, and we're not very good at explaining it afterwards either

References

• Hao et al., "Training Large Language Models to Reason in a Continuous Latent Space" (Coconut, Meta, December 2024): https://arxiv.org/abs/2412.06769
• Korbak et al., "Chain of Thought Monitorability: A New and Fragile Opportunity for AI Safety" (July 2025): https://arxiv.org/abs/2507.11473
• Hacker News discussion of Hybrid Reasoning Agents: https://news.ycombinator.com/item?id=50145337

Related

• Hybrid Reasoning Agents
• AI Reasoning Models
• Chain-of-Thought (CoT) prompting
• AI Safety
• AI Alignment
• Embeddings
• System 1 and System 2 Thinking
• Large Language Models (LLMs)

About Sébastien

Ready to get to the next level?