Glonce

Jev detects AI alignment failures zero-shot, 63x cheaper than LLM judges

researchThe Glonce editorsHugging Face🔥78Glonce rating78Impact63Novelty66Relevance95Surprise48Players32Credibility87
Jev detects AI alignment failures zero-shot, 63x cheaper than LLM judges

Most tools that screen deployed language models for alignment failures are either generative judges, which spend a full decoding pass on every criterion they check, or classifiers that read token probabilities, such as Llama Guard, which still score only one fixed label per call. Jev is a model trained with reinforcement learning for calibrated decisions (RLCD) that instead answers many typed questions about a single input with calibrated probabilities in one call. Whether Jev actually detects alignment failures had not been measured before this work.

The authors present RLCDAlignBench, which benchmarks Jev on ten alignment failures: sycophancy, jailbreaks, deception, prompt injection, hallucination, privacy violation, social bias, reward hacking, concealing uncertainty, and power seeking. The benchmark spans 44 benchmarks and five target models, with labels coming from each benchmark's own scorer, except on two benchmarks where humans supplied the labels. Many of these failures are relational: they are defined against a reference, such as the user's belief or an injected instruction, that the model's response alone does not reveal. The paper's key idea is to vary what Jev is asked (the question's wording and answer type) separately from what it is shown (the fields of the input).

A single generic question reaches a median AUROC of 0.886 zero-shot and beats supervised baselines on most of the benchmarks. Question wording turns out to matter little, while the context Jev sees matters more, mainly through input fields that encode the label. Overall, Jev matches the reference scorer's agreement with human labels, surfaces label defects in existing benchmarks, and costs 63x less than LLM-judge scorers. Code and data for RLCDAlignBench are published on GitHub.

Key facts

  • Jev, a model trained with reinforcement learning for calibrated decisions (RLCD), is tested for the first time as a zero-shot detector of alignment failures
  • RLCDAlignBench covers ten failure types, sycophancy, jailbreaks, deception, prompt injection, hallucination, privacy violation, social bias, reward hacking, concealing uncertainty and power seeking, across 44 benchmarks and five target models
  • Labels come from each benchmark's own scorer, except on two benchmarks where humans provided the labels
  • A single generic zero-shot question reaches a median AUROC of 0.886 and beats supervised baselines on most benchmarks
  • Jev matches the reference scorer's agreement with human labels and costs 63x less than LLM-judge scorers; code and data are published on GitHub

Why it matters

Current alignment-failure detectors are either generative judges that spend a decoding pass on every criterion or fixed-label classifiers like Llama Guard that score one label per call. Jev answers many typed questions about a single input with calibrated probabilities in one call, and this paper is the first to measure whether that capability actually catches alignment failures, at far lower cost than LLM-judge scorers.

Who it affects

Teams building safety and alignment evaluation pipelines, benchmark maintainers who need scalable labeling, and anyone deploying language models who wants to screen for the ten failure types covered: sycophancy, jailbreaks, deception, prompt injection, hallucination, privacy violation, social bias, reward hacking, concealing uncertainty, and power seeking.

How to use it

Code and data for RLCDAlignBench are published at github.com/sumleo/RLCDAlignBench. The approach separates what Jev is asked, a single generic question works well since wording matters little, from what it sees, the input's fields, which matter more when they encode the label. That lets practitioners apply Jev zero-shot without building a supervised training set for every new benchmark.

How solid is it

The benchmark spans 44 benchmarks and five target models. Labels come from each benchmark's own scorer, except on two benchmarks where humans supplied the labels. A single generic question reaches a median AUROC of 0.886 zero-shot, beating supervised baselines on most benchmarks, and Jev matches the reference scorer's agreement with human labels. The source reports only this median figure rather than a per-failure-type breakdown, and does not name the authors, institution, or publication date.

Risks and caveats

Only two of the 44 benchmarks carry human labels; the rest are checked against each benchmark's own scorer, which risks circularity if that scorer shares Jev's blind spots. The source gives median AUROC only, not results broken down by individual failure type, and does not explain the mechanics of the RLCD training procedure beyond naming it.

“A single generic question reaches a median AUROC of 0.886 zero-shot and beats supervised baselines on most benchmarks.”

— RLCDAlignBench paper