Glonce

Sparse autoencoder features are only partly additive, paper finds

researchThe Glonce editorsHugging Face🔥58Glonce rating58Impact45Novelty60Relevance95Surprise30Players12Credibility75
Sparse autoencoder features are only partly additive, paper finds

Sparse autoencoders (SAEs) describe a model's activations as a set of sparse, linearly decoded features. A paper on Hugging Face asks a question that this picture leaves open: do the downstream effects of those features add up? The authors say they need not.

To measure it, they introduce SAE Additivity Residuals (SAR). SAR captures the behavioral variation left unexplained above a chosen interaction order. Under independent feature subset sampling, the authors state that population SAR is exactly the normalized energy of interactions above that order, and that it bounds the coalition ordering error of the best low order approximation. In other words, it tells you how much of a group of features' joint effect a simple model of a given order cannot account for.

The next problem is separating compositionality that comes from the SAE representation itself from compositionality that comes from the downstream computation. To do that, the authors compare learned SAE coordinates with sum preserving random rotations of the same selected subspace. The learned coordinates show lower first order residuals than the rotated ones across all 18 model and behavior settings. So the SAE basis is more additive than an arbitrary basis of the same subspace.

But it is not fully additive. Pairwise terms improve held out coalition ordering in every setting, and they outperform parameter matched models that use randomly selected third order terms. In a combined steering experiment, pairwise modeling raises rank fidelity from 0.5874 to 0.7790 and reduces ordering error from 0.1444 to 0.0606.

The authors' conclusion is that SAE coordinates support comparatively compositional behavioral descriptions, while faithful joint explanations and interventions still require interactions beyond individual feature effects. Code is available at https://github.com/OpenSAE/SAE-Behavioral-Compositionality.

Key facts

  • The paper introduces SAE Additivity Residuals (SAR), a measure of behavioral variation left unexplained above a chosen interaction order.
  • Learned SAE coordinates show lower first order residuals than sum preserving random rotations of the same subspace in all 18 model and behavior settings.
  • The coordinates are still not fully additive: pairwise terms improve held out coalition ordering in every setting and beat parameter matched models with randomly selected third order terms.
  • In a combined steering experiment, pairwise modeling raises rank fidelity from 0.5874 to 0.7790 and cuts ordering error from 0.1444 to 0.0606.
  • Code is released on GitHub at OpenSAE/SAE-Behavioral-Compositionality.

Why it matters

SAEs are a common tool for breaking model activations into human-readable features, and explanations built on them often treat each feature's effect as separate. This paper gives a way to quantify how far that assumption holds. Its answer is a middle one: SAE features are more compositional than random directions of the same subspace, yet their joint effects contain interactions that single-feature accounts miss. That bears directly on how far a list of active features can explain behavior or guide an intervention.

Who it affects

Mainly researchers who use SAE features to explain model behavior or to steer it. The finding is that faithful joint explanations and interventions still require interactions beyond individual feature effects, so anyone combining several features at once is in scope.

How to use it

SAR can serve as a check on how much of a group of features' combined effect a first order (additive) description leaves out, at whatever interaction order you choose. The authors' results suggest adding pairwise terms when ordering or predicting the effect of feature coalitions; in their combined steering experiment this lifted rank fidelity from 0.5874 to 0.7790. The code is at https://github.com/OpenSAE/SAE-Behavioral-Compositionality.

How solid is it

The source is a paper abstract, and the headline results are quantified: a consistent direction across all 18 settings for the comparison with random rotations, and pairwise gains in every setting. The SAR identity is stated as holding under independent feature subset sampling, so it is a conditional result. The code is public, which allows others to check the work.

Risks and caveats

The abstract does not name the models, SAEs, or behaviors used in the 18 settings. It gives no magnitudes for the lower first order residuals of learned coordinates versus random rotations, and it does not say the results hold for production-scale or frontier models. The rank fidelity and ordering error figures come from one combined steering experiment, and the abstract does not say what the baseline is beyond calling the gains the effect of pairwise modeling. The authors' own framing is also modest: the coordinates are "comparatively" compositional, not fully so.