Glonce

The right principal components narrow deception probes' generalization gap

researchThe Glonce editorsarXiv🔥75Glonce rating75Impact70Novelty75Relevance100Surprise55Players40Credibility80

Linear probes are a common interpretability tool: train a simple linear classifier on a language model's internal activations and it can flag whether the model is exhibiting some target behavior or concept, deception among them, without relying on what the model actually says. Their known weakness is that they often fail to transfer: a probe trained to catch deception in one setting can silently stop working on a different, out-of-distribution setting it was never trained on.

The authors study this transfer problem directly. They train deception-detection probes on the activations of Llama-3.1-8B-Instruct and test them on three held-out datasets the probes never saw during training. Rather than probing on the full activation space, they project the inputs onto a small subset of principal components drawn from the training data's own activation distribution. That projection alone lets a probe trained on one dataset transfer to another, nearly matching the performance of a probe trained directly on the target data itself.

Not every principal component in that subset generalizes equally well, so the authors add a second step: an LLM judge is shown each component's most and least activating examples and asked whether the pattern implies a deception direction that would hold up across datasets. Keeping only the highest-scoring components, and probing on those alone, closes what the authors call the baseline-to-oracle gap, the gap between a probe transferred as-is from another dataset and one trained directly on the target dataset, by 78% on a benchmark named Insider Trading Report and by 25% on one named Sandbagging.

Examining what the surviving directions actually encode, the authors find a pattern: the directions a probe trained on one dataset weights most heavily tend to capture surface features specific to that dataset, details that do not carry over elsewhere. The directions that do transfer instead encode the same underlying deception contrast in a more abstract form, one that can be captured in a natural-language description, which is exactly what lets an LLM judge read and select them. The authors take this as evidence that a probe's ability to generalize out of distribution is largely determined by which subspace of the activation space it operates in, rather than by other aspects of how it was trained.

Key facts

  • Probes trained to flag deception in Llama-3.1-8B-Instruct's activations were tested for transfer across three held-out deception-detection datasets they never saw during training.
  • Projecting the probe's inputs onto a small subset of principal components from the training data's own activation distribution lets it transfer to a new dataset, nearly matching a probe trained directly on that target data.
  • Having an LLM judge each component's most and least activating examples, then keeping only the highest-scoring components, closes the baseline-to-oracle performance gap by 78% on the Insider Trading Report dataset and by 25% on Sandbagging.
  • The directions a probe relies on most heavily within its own training dataset tend to encode surface features specific to that dataset; the directions that actually transfer encode the same deception contrast more abstractly, in terms a natural-language description can capture.
  • The authors conclude that a probe's ability to generalize out of distribution is largely determined by which subspace of the activation space it is restricted to, not simply by how it was trained.

Why it matters

Researchers building deception detectors out of a language model's internal activations, rather than trusting what the model says, have run into a basic obstacle: a probe that works on the data it was trained on can quietly stop working the moment it meets a new kind of deception it was never shown. This paper reframes that failure as a question of which subspace the probe operates in. It finds that the activation space contains both a genuinely transferable deception signal and source-specific surface detail that does not generalize, and that restricting the probe to the right subset of principal components is what recovers most of the lost performance. It also shows that an LLM can identify which components carry the transferable signal simply by reading what each one represents. The authors describe this, broadly, as evidence that a probe's out-of-distribution robustness is largely determined by subspace selection.

Who it affects

AI safety and interpretability researchers who use linear probes to monitor a model's internal state for deception, rather than relying on its outputs, are the direct audience. Two of the paper's three held-out test datasets, named Insider Trading Report and Sandbagging, are the kind of deception-detection benchmarks such evaluation work is built on. Anyone assembling a probe-based safety check faces the same open question this paper addresses: will a probe trained on the available data hold up on deceptive behavior it has not yet seen.

How to use it

The recipe: run the source probe's training-distribution activations through a principal component analysis, then, instead of probing on the full activation space or an arbitrary subset, show an LLM judge each component's most and least activating examples and ask it whether that pattern implies a deception direction that would generalize. Keep only the highest-scoring components and probe on that subspace alone. On the paper's two reported benchmarks this closed 78% (Insider Trading Report) and 25% (Sandbagging) of the gap between a probe transferred as-is from another dataset and one trained directly on the target data.

How solid is it

The evidence comes from a single base model, Llama-3.1-8B-Instruct, evaluated across three held-out datasets; the abstract reports concrete percentage figures for two of them, Insider Trading Report and Sandbagging. The first-stage finding, that projecting onto a subset of principal components alone nearly matches oracle performance, is described qualitatively rather than with a figure. Even the LLM-judge-refined method only narrows the baseline-to-oracle gap by 78% and 25%, rather than closing it outright, and those are relative reductions in a gap whose absolute starting size is not stated, so the same percentage could reflect a large or a small real change in accuracy. The abstract does not state a publication venue, date, or timeline.

Risks and caveats

The refinement narrows the baseline-to-oracle gap; it does not close it. A 78% and 25% reduction still leaves a real shortfall on both benchmarks, which matters if the probe is meant to catch deception a model would otherwise get away with. That refinement also leans on an LLM judge's own reading of a handful of extreme-activating examples per component, so the method's reliability now depends in part on how well that judge's interpretation lines up with the true, generalizable signal rather than a superficial pattern in those particular examples.

“Broadly, our results suggest that the OOD robustness of probes is largely determined by subspace selection.”

— the paper