LLMs know the hidden constraint but fail to use it

Large language models often stumble on a specific kind of prompt: one where a salient surface cue points toward one answer while an implicit feasibility constraint, something never stated outright, points toward another. A new paper argues that measuring this with plain aggregate accuracy hides what is actually going on, because a model can land on the correct answer either by genuinely inferring the hidden constraint or by defaulting to a cautious answer regardless of it. The two look identical in a raw accuracy score.

To separate them, the authors introduce conditional constraint activation, a four-part diagnostic they call a quartet. Knowledge asks whether the constraint is encoded inside the model at all. Symmetry checks whether that internal encoding looks the same whether the constraint is present or absent from the prompt. Routing asks whether the internally encoded constraint is actually used when the model produces its decision. Repair tests, causally, whether a donor activation can fix a case where routing failed.

Running this quartet across 14 models turns up two distinct failure modes. On two of those models, both open weight, the authors go a step further and probe the internal activations directly: the probes decode the hidden constraint at above 88% accuracy, confirming the information is present internally even in cases where the model's answer ignores it. Activation patching, transplanting a donor activation to see whether it repairs a broken routing path, repairs one of the two models (+6.4 nats) but not the other (-0.07).

The authors then map what they call a mitigation frontier: a set of prompting-based interventions meant to help a model act on the constraint it already has. None of them reach the point on that frontier that direct activation repair reaches. Instead, every prompted intervention tested inflates the model's conservative bias, and all of them do it through the same channel, explicitly spelling out the prerequisite in the prompt rather than leaving it implicit. The authors' conclusion is blunt: hidden-constraint failure is a routing problem, not a knowledge problem.

Key facts

A four-part quartet diagnostic, covering Knowledge, Symmetry, Routing, and Repair, applied across 14 language models finds two distinct failure modes in how models handle a hidden constraint that competes with a salient surface cue.
On two open-weight models, internal probes decode the hidden constraint from activations at above 88% accuracy, showing the constraint is encoded internally even when a model's output ignores it.
Activation patching with a donor activation repairs the routing failure in one of the two models (+6.4 nats) but not the other (-0.07).
None of the prompted interventions tested reach the level of repair that direct activation patching achieves; every one of them instead inflates the models' conservative bias.
All the tested prompted interventions inflate conservative bias through the same channel, explicitly mentioning the prerequisite in the prompt, and the authors conclude hidden-constraint failure is a routing problem, not a knowledge problem.

Why it matters

The usual assumption is that when a model misses an implicit constraint, it simply does not know about it. This paper's diagnostic suggests otherwise, at least in the cases tested: the constraint is often already encoded internally with high fidelity, and what fails is getting that internal signal to influence the output. That distinction changes where a fix should be aimed. A knowledge gap calls for more training data or clearer instructions. A routing gap is a mechanistic issue inside the model, and the paper's own mitigation-frontier results show that surface-level prompting does not reliably close it.

Who it affects

The findings target researchers working on LLM interpretability, evaluation and safety, and anyone building applications where a model needs to respect an unstated feasibility limit rather than just react to whatever cue is most salient in the prompt. The behavioral diagnostic covers 14 models, and the deeper internal analysis covers two open-weight models, so the pattern is presented as a general one rather than a quirk of a single system.

How to use it

There is no product or release attached to this paper, but it offers a diagnostic method: the Knowledge, Symmetry, Routing, and Repair quartet, which can be used to check whether a model's failure on an implicit constraint is a knowledge gap or a routing gap before choosing a fix. The mitigation-frontier results are a caution against reaching for prompt engineering as that fix. Every prompted intervention the authors tested made the tested models more conservative rather than actually repairing how the constraint reaches the decision, and none matched what direct activation repair achieved on the model where repair worked.

How solid is it

The behavioral half of the diagnostic runs across 14 models, giving the two-failure-mode finding a reasonably broad base. The mechanistic half, decoding the constraint from internal activations and testing whether a donor activation repairs routing, rests on only two open-weight models, and even there the result is split: repair worked on one model (+6.4 nats) and failed on the other (-0.07). That split is itself a finding, since it shows the routing failure does not behave the same way across architectures, but it also means the causal repair claim has only been demonstrated on a single model so far, not shown to generalize.

Risks and caveats

This account is drawn from the paper's abstract alone. It does not name the authors, their institutions, or which 14 models or which two open-weight models were used, and it gives no worked example of what an actual surface-cue-versus-feasibility-constraint prompt looks like, so the concrete task remains abstract here. No publication date is given either. The -0.07 repair figure carries no stated unit in the source, unlike the +6.4 figure, which is explicitly given in nats, and none of the reported numbers, the decoding accuracy above 88%, the +6.4 figure or the -0.07 figure, are downstream task-accuracy results or come with a stated baseline; they are internal probing and patching diagnostics, a narrower claim than showing the model gets the task right.

“Hidden-constraint failure is a routing problem, not a knowledge problem.”

— the paper's authors