Sparse autoencoders encode parts of speech as distributed latent groups, study finds
A study of Sparse AutoEncoders (SAEs), a popular tool for peering inside language model representations, asks a basic question about how much of what SAEs expose is actually grammatical structure rather than lexical shortcuts. The researchers used part-of-speech (PoS) categories, such as nouns, verbs and other grammatical classes, as a controlled test case to check whether morpho-syntactic information lives in individual SAE latents or is spread across structured groups of features. They found that PoS distinctions are highly recoverable from SAE activations, but they do not correspond one-to-one with individual latents: there is no single 'noun latent' or 'verb latent' to point to. Crucially, this recoverability is not simply a matter of the model memorising which words belong to which category, since lexical memorisation alone cannot account for it. The study also reports that Open PoS classes (categories like nouns and verbs that readily admit new words) and Closed PoS classes (categories such as pronouns or conjunctions with fixed, limited membership) differ substantially in how they are represented. Instead of atomic features, each PoS category is supported by a compact group of sparse latents, and the size and composition of these groups vary substantially from tag to tag. These latent groups are stable when tested on held-out data not used to identify them, yet they also overlap with the groups supporting related categories. The overall conclusion is that SAEs localise morpho-syntactic information in a distributed, category-dependent form rather than through single, atomic grammatical features.
Key facts
- The study uses part-of-speech categories as a controlled test case for what SAE latents actually encode.
- PoS distinctions are highly recoverable from SAE activations but do not map one-to-one onto individual latents.
- This recoverability is not reducible to lexical memorisation, and Open PoS classes differ substantially from Closed PoS classes.
- Each category is supported by a compact group of sparse latents whose size varies substantially across tags.
- These latent groups stay stable on held-out data while overlapping with groups for related categories.
Why it matters
SAEs are widely used to make language model internals interpretable, on the assumption that individual latents correspond to clean, human-readable concepts. This study tests that assumption against a well-defined linguistic structure, parts of speech, and finds the picture is messier: grammatical categories are encoded collectively across groups of latents rather than by single dedicated units. That complicates a common shortcut in interpretability work of reading one latent as one concept.
Who it affects
The finding is aimed at researchers building or relying on SAEs to interpret language model representations, including anyone using SAE latents to explain or intervene on model behaviour related to syntax or grammar.
How to use it
For interpretability researchers, the practical takeaway is to look for groups of co-active latents rather than searching for a single latent per grammatical category when studying morpho-syntactic information in SAEs, and to expect Open and Closed PoS classes to behave differently rather than uniformly.
How solid is it
The account here is based on the paper's own description of its method and findings. The source text does not name the authors, the institution, the language or dataset used, the specific model or SAE architecture, or any quantitative results such as accuracy or latent counts, so those specifics cannot be reported.
Risks and caveats
Without named authors, a specified dataset or language, or reported numbers, the strength and generality of the claims cannot be independently assessed from what is available here; the overlap between related categories' latent groups also means the boundaries between categories are not sharp.
“SAEs localise morpho-syntactic information in a distributed and category-dependent form rather than through atomic grammatical features.”
— the paper