Study maps how LLMs geometrically separate reasoning steps

A research team examined how large language models internally organize the different functional steps of chain-of-thought reasoning, such as formulating a problem, decomposing a goal, and performing deduction. These operations are visibly distinct in the text a model produces, but it was unclear whether they are also organized as distinct geometric structures inside the model's hidden representations.

The researchers tested whether reasoning operations are separable in held-out representations, meaning representations not used to build the analysis, and found that they are, with separability peaking in the middle layers of the model rather than at the start or end. They then checked whether this separation could simply be an artifact of which words appear or where tokens sit in the sequence, and verified that it is not explained by such lexical or positional confounds: the structure reflects the reasoning operation itself, not surface-level patterns.

Looking across layers, the team found that token-by-token alignment with a given operation becomes more distributed over longer spans of text as depth increases. They also found that identical surface tokens are represented differently depending on which operation the surrounding chunk of reasoning belongs to, meaning the same word can carry a different internal representation depending on its functional role in the reasoning process.

Using attention-masking interventions, in which parts of the model's attention are selectively blocked, the researchers further showed that operation-aligned representations at the start of a reasoning chunk depend on the reasoning context that came before it, rather than forming independently. Taken together, the authors conclude that the models maintain a correspondence between the linguistic reasoning expressions in their output and internal geometric structures. Code and project materials for the work have been released on GitHub under the naver-ai organization, at the project titled 'beneath-cot'.

Key facts

Reasoning operations such as problem formulation, goal decomposition, and deduction are separable in held-out hidden representations of large language models.
Separability between operations peaks in the middle layers of the model, not at the earliest or latest layers.
The separation is not explained by lexical or positional confounds, meaning it reflects the reasoning operation rather than surface word choice or token position.
Identical surface tokens are represented differently depending on the reasoning operation of the chunk they appear in.
Attention-masking interventions show that operation-aligned representations at the start of a reasoning chunk depend on the preceding reasoning context.

Why it matters

Chain-of-thought reasoning is usually studied by reading the text a model outputs, treating the words as a transparent trace of what the model is doing. This work instead asks whether that textual structure has a counterpart inside the model's hidden representations. Finding that reasoning operations occupy geometrically separable regions of representation space, and that this separation is strongest in the middle layers, suggests the model is not just producing reasoning-shaped text but organizing an internal structure that tracks the functional role each part of the reasoning plays.

Who it affects

The findings are most directly relevant to researchers working on mechanistic interpretability and on understanding or steering chain-of-thought reasoning in large language models. They also matter to anyone building tools that probe, monitor, or intervene on a model's internal states during reasoning, since the results indicate that a token's representation depends on its surrounding reasoning context rather than being fixed by the token alone.

How to use it

The authors have released code and project materials on GitHub under the naver-ai organization (project 'beneath-cot'), which lets other researchers reproduce the separability analysis and attention-masking interventions or apply the same methodology to other models and reasoning tasks.

How solid is it

The claims rest on representation-level analysis of held-out data plus targeted attention-masking interventions, which is a standard mechanistic interpretability approach for testing whether an effect is causal rather than merely correlational. The source text does not specify the particular models, benchmarks, or datasets used, nor does it report numeric separability scores, so the strength of the effect and how it generalizes across model families cannot be assessed from the available text.

Risks and caveats

The abstract does not name the authors, their institutions, or a publication venue, and gives no quantitative separability figures or details of the probing method beyond the general description of held-out representations and attention-masking interventions. Readers wanting a fuller technical assessment should consult the paper itself and the released code rather than relying on the abstract alone.

“language models maintain representational correspondence between linguistic reasoning expressions and their internal geometric structures”

— the authors