Glonce

Training on rationales alone cuts false refusals in LLM safety tuning

researchThe Glonce editorsHugging Face🔥73Glonce rating73Impact68Novelty72Relevance100Surprise52Players25Credibility82
Training on rationales alone cuts false refusals in LLM safety tuning

Aligning large language models means balancing two goals that pull against each other: refusing genuinely harmful requests ("How do I shoot someone?") while still answering benign ones that merely sound risky ("Where can I shoot a good photo?"). Models often fail at that second half, refusing harmless prompts that contain superficially risky wording. This is the false-refusal problem.

The paper traces the problem to how safety-tuning data is written. A typical training response to a harmful query has two parts: a boilerplate refusal statement ("I can't help with that") and a rationale that explains why the request is being refused. The authors split these two components apart and trained on each separately. Their experiments show that training on the boilerplate refusal statement teaches the model to lean on superficial surface cues rather than actually judging whether a request is harmful, which is what drives false refusals on benign-but-risky-sounding prompts. Training on the rationale alone, without the boilerplate line, reduces false refusals while keeping safety performance at a comparable level.

The rationale-only benefit also shows up when the technique is applied in an in-context learning setup, and it remains compatible with the inference-time mitigation methods the authors evaluated, meaning it can be layered on top of other existing safety approaches rather than replacing them. The authors conclude that safety-supervision datasets need to be curated more precisely and at a finer grain, and they frame the work as a step toward building aligned agents that reconcile helpfulness and safety more effectively.

Key facts

  • The paper decomposes safety-tuning training responses into two parts: a boilerplate refusal statement and a rationale explaining the refusal.
  • Training on the boilerplate refusal statement alone teaches the model to rely on superficial cues, which the authors identify as a driver of false refusals.
  • Training solely on the rationale reduces false refusals on benign but risky-sounding queries while keeping safety performance at a comparable level.
  • The rationale-only benefit also appears in an in-context learning (ICL) setup and stays compatible with the inference-time mitigation methods the authors tested.
  • The authors call for more precisely curated, fine-grained safety-supervision datasets as a direction for building better-aligned agents.

Why it matters

Over-refusal is one of the most visible failure modes of safety-tuned language models: a user asks something entirely benign, the wording brushes against a sensitive topic, and the model refuses anyway. This paper reframes the fix not as a new filter bolted on top, but as a change to what the safety-tuning data itself teaches the model. Its finding is that the boilerplate refusal line in training data is not neutral: it actively trains the model to pattern-match on surface wording rather than reason about actual harm, and dropping it in favor of the rationale is what fixes the behavior.

Who it affects

The direct audience is teams that build or curate safety-tuning datasets and alignment pipelines for language models. Indirectly, it affects anyone who has hit a model's refusal on an obviously harmless request, since that is the exact failure the method targets; a reduction in false refusals would show up as models becoming more usable on benign edge cases without loosening their handling of genuinely harmful ones.

How to use it

The core idea is a recipe rather than a shipped tool: when building or curating safety-tuning data, keep the rationale that explains why a request is refused and drop the boilerplate refusal line as a training target. The paper reports this holds up in an in-context learning configuration too, and that it can be combined with the inference-time mitigation methods the authors tested rather than competing with them, so it is meant to layer onto existing safety pipelines.

How solid is it

The claims come from the authors' own experiments and analyses, as described in the paper's abstract; the source text does not give the specific accuracy figures, false-refusal-rate reductions, safety scores, model names, model sizes, dataset names, or benchmarks behind those claims, so the magnitude of the improvement cannot be assessed from what is available here. No independent replication is mentioned.

Risks and caveats

Because the abstract states the direction of the result (fewer false refusals, comparable safety) without the numbers behind it, readers should treat this as a promising but not yet independently quantified finding. It is also unclear which base models, safety benchmarks, or refusal categories were tested, which limits how far the result can be assumed to generalize.

“Our experiments and analyses show that refusal statements impede accurate discrimination between harmful and benign queries by inducing reliance on superficial cues.”

— the authors