Glonce

PCA finds stylistic axes in LLM activations, but only in some models

researchThe Glonce editorsarXiv🔥72Glonce rating72Impact60Novelty62Relevance100Surprise68Players45Credibility82

A new method finds stylistic dimensions inside a large language model's hidden activations without any training or labeled data. The approach: repeatedly sample completions of a single prompt at elevated temperature, apply Principal Component Analysis (PCA) to the pooled hidden activations across those samples, then automatically label the resulting axes from the generations sitting at each axis's pole.

The authors validate the discovered axes against 245 human-elicited stylistic annotations gathered in a two-phase study. On their strongest model, Qwen-3.5-4B-Instruct, the top two discovered axes match the stylistic dimensions people spontaneously ask for with 72.8% precision and 43.6% macro-recall. Separately, human raters judge 75.6% of the axes' polar generations to accurately match their automatically generated labels, and those validity ratings show 90.9% adjacent inter-annotator agreement.

The result is strongly model-dependent. Qwen models and Llama-3.2-3B all expose axes that line up with human-salient style dimensions. DeepSeek-7B-Chat does not: its precision drops to 35.3%, and its leading principal components are dominated by structural variance rather than genuine stylistic variance. The authors read this as evidence that simple PCA over a model's own decoding variance is a cheap, effective probe of stylistic structure in LLM representations, one that also exposes real differences in how different models organize that structure internally.

The abstract does not name the paper's authors or institutions, does not release the 245-annotation dataset or code, does not put a number on how the method compares to supervised contrastive baselines, and gives no indication of how long sampling and PCA take to run.

Key facts

  • The method needs no training or labeled data: sample completions of one prompt at elevated temperature, run PCA on the pooled hidden activations, and auto-label the resulting axes from the pole generations.
  • Validated against 245 human-elicited stylistic annotations; on the strongest model, Qwen-3.5-4B-Instruct, the top two axes match human-requested dimensions with 72.8% precision and 43.6% macro-recall.
  • 75.6% of validity ratings judge the axes' polar generations accurate to their labels, with 90.9% adjacent inter-annotator agreement.
  • Discoverability is model-dependent: Qwen models and Llama-3.2-3B expose human-salient axes, but DeepSeek-7B-Chat drops to 35.3% precision, its leading components dominated by structural rather than stylistic variance.
  • The abstract names no authors or institutions and gives no code/data release, no quantified comparison to supervised baselines, and no timing figures.

Why it matters

Finding which stylistic dimensions matter in a model's output usually needs supervised contrastive data built for that purpose. This method skips that step entirely: it only needs a model to generate multiple completions of a prompt at high temperature, then PCA on the resulting hidden activations does the rest. That makes probing a model's stylistic structure cheap enough to run per-prompt rather than as a separate research pipeline.

Who it affects

The audience is interpretability and alignment researchers who study how LLMs represent style internally, plus anyone building tools for controllable generation or style transfer who would otherwise need labeled contrastive examples to find useful directions in activation space.

How to use it

The recipe as described: pick a prompt, sample many completions at elevated temperature, pool the hidden activations from those completions, run PCA on the pool, and label the axes automatically using the generations that sit at each axis's extremes (poles). No fine-tuning or external labeled data enters the pipeline.

How solid is it

The authors ran a two-phase human validation against 245 stylistic annotations. On the best-performing model, Qwen-3.5-4B-Instruct, the top two axes matched human-requested stylistic dimensions with 72.8% precision and 43.6% macro-recall, and 75.6% of validity ratings called the axes' polar generations accurate to their labels, with 90.9% adjacent agreement between annotators. The abstract does not quantify how this compares to supervised contrastive baselines, only that the method avoids needing them.

Risks and caveats

Discoverability is strongly model-dependent. Qwen models and Llama-3.2-3B exposed human-salient axes, but DeepSeek-7B-Chat's precision fell to 35.3%, with its leading principal components dominated by structural variance rather than actual style. The abstract names no authors or institutions, releases no code or the 245-annotation dataset, and gives no timing figures for how long sampling and PCA take.