Essay argues neural networks are already clean, superposition was misread

In a first-person essay titled "Neural networks are clean", an interpretability researcher says that after many months studying neurons and circuits in image and language models, they now believe a network's own mechanisms are its legible building blocks, and that the outlook for mechanistic interpretability may be far better than they had feared. The central claim: the canonical evidence for polysemanticity and superposition, which convinced much of the field (including the author) that networks pack concepts arbitrarily, was misread. What looked like arbitrary packing was mostly a motif the author calls recursive generalization, which is much less concerning.

The author frames two approaches to mechanistic interpretability. The first is normal science on the network itself: look at neurons and circuits, study how they relate, watch data flow through them, and run experiments at agentic scale. The second starts from a list of hypothesized features and tries to extract them all with ever more sophisticated reconstruction objectives, usually ignoring the original network. The field began with the first and fell back to the second because networks seemed to be tangled knots. The famous example was an image-model neuron that responded to both cat faces and the fronts of cars, and the last layer of InceptionV1 seemed so full of arbitrary mixtures that almost none of its neurons were easily interpretable.

Superposition was the leading explanation. It proposed a second learning principle: when a network runs out of neurons, it stops grouping related things and folds unrelated things together because they do not interfere. The result is a knot that must be unfolded first, in practice by training a second model such as a sparse autoencoder to approximate the network as it would look unfolded.

The author's main result is that superposition was the wrong explanation for InceptionV1. Architectural problems made later layers overflow, and weak interpretability tools amplified the problem. Later architectural changes designed to help deep layers learn better had the unintended effect of making them more interpretable. With these ordinary improvements, and no change to model size or training data, 99-100% of neurons in the last layer are easily interpretable and the model is broadly legible throughout. The author calls it the first non-trivial neural network they feel they mostly understand, and says versions trained across a 300× range in size are legible at every scale. With less room, neurons cover coarser concepts (one car neuron for many kinds of car); with more room, the neuron differentiates into specific cars. So the failure was an ordinary failure of the first learning principle, and it is easy to fix.

The canonical vision example was also an error, the author says. In 2019, the author and Chris Olah found the circuit that was introduced as superposition in "Zoom In": a car feature appearing to spread into a set of unrelated dog neurons. It was the only example shown. They read it as the network hiding the car in a high-dimensional space for downstream neurons to read back out, but nothing reads that combination; it was ordinary generalization and composition. To the author's knowledge, no one has published a second circuit with the right shape.

On language models, the author says they are far more interpretable than generally believed. Anthropic's 2023 Towards Monosemanticity, which studied a small one-layer model, gave the median neuron 0 out of a possible 14 on its rubric, meaning the rater could not form any hypothesis. The 2023 OpenAI automated interpretability project, which the author started, found about 1.7% of neurons interpretable by its most generous measure. With better tools and modern models, the author says the picture looks different. In preliminary audits of Qwen3-32B, they estimate roughly 90% of neuron poles have one main job, often with small related offshoots; another ~7% have two related jobs; and only ~2-4% are not yet understood. A note explains that the share depends on the threshold for weak activations, depth, density of sub-circuits and what counts as "one thing". For example, weak activations of a Trump neuron bring in MAGA hats and related people, and a Titanic neuron's apparent "unrelated jobs" were often predicting that "boat" comes next, which the tool missed because it only looked backward. In smaller sets examined more closely, considering strong activations, the author gets closer to 97% non-arbitrary mixtures, though the samples are relatively small. The author says the single cleanliness score is only a broad pointer to the regime a network is in. Dense and Mixture of Experts models seem roughly similar, and similar results were found in models up to trillions of parameters, like Kimi K3, which a companion paper will cover in more detail.

Most remaining apparent polysemanticity seems to be recursive generalization: as a neuron grows, it sprouts offshoots the way a branch sprouts sub-branches. One neuron that detected "iy" grew to detect "diy" and then "do it yourself", which does not contain "iy". The idea first appeared in a forgotten subsection of the 2021 Multimodal Neurons paper. The author says that at the time they had not watched neurons grow over training; now it is easier, and it holds up everywhere they have looked. Even the original InceptionV1 was not knotted: retraining the original architecture and following the least legible neurons shows them overgrowing locally into reasonable, adjacent territory.

The author helped propose both explanations, which are in some ways opposites: generalization groups similar things, superposition groups unrelated things. The author now thinks recursive generalization was basically right, leaving one learning principle, with generalization turning out to be fractal. That is less concerning, since there is something to learn from how things were grouped, and explanations can be fractal too: start basic, then zoom in.

Why was the gap so large? The author names several compounding problems: 2023 explainer models could not run the loop of conjecture and test; tools and metrics rewarded explanations that simulate activations rather than ones that are right; nobody knew which neuron motifs to expect in language models; and most studies used small or simplified models that really had run out of room. Broad neurons are usually easy to break down once you know what to look for. The author also argues interpretability has momentum like a jigsaw puzzle: once the map starts to fill in, each piece supplies clues, so "nearly empty" and "mostly filled in" are natural anchors. The visible text ends as the author begins an example of a neuron that tracks "a sense of righteousness when a bargain purchase turns out to work better than the expensive alternative", noting that neurons tracking feelings or social cues like this are among the most common found.

Key facts

The essay argues the canonical evidence for polysemanticity and superposition was misread; most apparent mixing is "recursive generalization", where a neuron's concept sprouts related offshoots as it grows.
After ordinary architectural improvements, with no change to model size or training data, 99-100% of neurons in the last layer of the modified InceptionV1 are easily interpretable, up from almost none originally; versions across a 300× size range stay legible.
Anthropic's 2023 one-layer study gave the median neuron 0 out of 14, and OpenAI's 2023 automated project found about 1.7% of neurons interpretable; preliminary Qwen3-32B audits estimate roughly 90% of neuron poles have one main job, ~7% two related jobs, ~2-4% not yet understood.
The author says similar results hold up to trillions of parameters, like Kimi K3, with a companion paper to follow, and calls the single cleanliness score only a broad pointer.

Why it matters

Mechanistic interpretability has leaned heavily on the idea that networks are tangled knots that must first be untangled, for example with sparse autoencoders trained as a second model. If the author is right, that detour is unnecessary and researchers can study neurons and circuits directly. The essay also reframes the field's best-known evidence: the cat-and-car neuron and the car/dog circuit that was read as superposition. The author, who helped propose both superposition and recursive generalization, says the field bet on the wrong one.

Who it affects

Mainly interpretability researchers deciding between studying a network's own neurons and circuits and building feature-extraction pipelines on top of it. It also bears on anyone who has taken the 2023 results (a median score of 0 out of 14 for a one-layer model, about 1.7% of neurons interpretable in OpenAI's project) as a measure of how opaque language models are. The author says tooling and architecture choices matter: architectural changes that help deep layers learn better also made InceptionV1 more interpretable.

How to use it

The essay points to a working method: study neurons and circuits directly, watch how neurons grow over training, and explain them in layers, starting with a basic explanation and zooming in. The author says the modified InceptionV1 can be browsed online, and that a companion paper will cover the larger language-model results in more detail. The author also cautions against treating a single cleanliness score as something to optimize directly or hold too tightly.

How solid is it

This is an argument-driven, first-person essay, and the visible text is cut off mid-essay. The strongest evidence is concrete: a retrained InceptionV1 variant where 99-100% of last-layer neurons are easily interpretable, legible across a 300× size range. The language-model figures are weaker. The Qwen3-32B numbers (roughly 90% / ~7% / ~2-4%) are described as preliminary estimates that shift with the threshold for weak activations, and the 97% figure comes from relatively small samples. The visible text does not name the author or the author's current employer, mentions no peer review or independent replication, and does not explain how the Kimi K3 results were measured.

Risks and caveats

The author's wording is hedged: most remaining polysemanticity "seems to be" recursive generalization, and the author knows of no second superposition-shaped circuit only "to my knowledge". Results depend on where one sets thresholds for weak activations and on what counts as "one thing", and the author notes tools can miss context, as with the Titanic neuron that was predicting "boat" next. The headline numbers for large language models are estimates, and the claim for trillion-parameter models is deferred to a companion paper. The essay argues superposition was the wrong explanation for InceptionV1 and for the canonical example; it does not claim superposition is shown absent in every network.

“Neural networks are already clean.”

— From the essay "Neural networks are clean"