AI trained on AI-written peer reviews loses judgment diversity, study finds

Researchers built a chain of AI peer-review models to test what happens when AI-generated reviews feed back into training future AI reviewers. Starting from Llama 3.1 8B, they first fine-tuned a reviewer on official ICLR conference reviews from 2018 through 2023. They then trained four successor models on ICLR 2024 data, each using a different mixture of official human reviews and model-generated reviews.

The study found that as synthetic reviews made up a larger share of the training mixture, the resulting models' rating distributions compressed: scores clustered more tightly instead of spanning the fuller range seen in human-written reviews. The models' judgments also became less semantically diverse, both when different reviews of the same paper were compared and across the review corpus as a whole. The researchers name this pattern scientific-judgment collapse: a feedback loop in which later AI reviewers, trained in part on the outputs of earlier AI reviewers, produce increasingly uniform and less discriminating evaluations.

To address the problem, the team introduces TrustReviewer, an open-source system for generating peer reviews of AI and machine learning papers. It intervenes at two points. For training-time prevention, the core reviewer is trained in a single stage on a curated corpus built to reduce low-quality and semantically degenerate supervision. For test-time correction, a technique called paired activation steering is applied to further reduce residual tendencies toward collapsed judgments, without further training or additional expert annotation.

The abstract gives no numeric figure for the size of the rating compression or the diversity loss, reports no quantitative results for how much TrustReviewer improves matters, and states no release date, license or code location beyond describing the system as open-source.

Key facts

Researchers fine-tuned a reviewer model from Llama 3.1 8B on official ICLR reviews from 2018 through 2023.
Four successor models were then trained on ICLR 2024 data, each using a different mixture of official and AI-generated reviews.
Introducing synthetic reviews compressed rating distributions and reduced semantic diversity, both within reviews of the same paper and across the corpus, a pattern the authors call scientific-judgment collapse.
The team introduces TrustReviewer, an open-source system that curates training data in a single training stage and applies paired activation steering at test time, without extra training or expert annotation.
The abstract gives no numeric magnitude for the compression or diversity loss and no quantitative results for TrustReviewer's effectiveness.

Why it matters

LLMs increasingly act as reviewers or reviewer assistants in scientific peer review, and model-generated reviews are already entering public data that future models will train on. This is a controlled demonstration that the loop this creates is not neutral: an AI reviewer trained partly on other AI reviews learns to produce less varied, more compressed judgments than one trained on human reviews alone. Left unaddressed, that risks flattening peer review's ability to tell genuinely different papers apart.

Who it affects

Anyone in the AI and machine learning research pipeline that touches AI-assisted peer review: conference organizers such as ICLR, human reviewers who use AI tools as assistants, authors whose papers are scored by such systems, and builders of future automated review systems who might otherwise train on AI-generated review text without realizing what it does to judgment diversity.

How to use it

TrustReviewer is described as an open-source system rather than a priced product. It is meant to be built into an AI-assisted review pipeline at two points: its training-time step curates the corpus used to train the core reviewer model, and its test-time step applies paired activation steering to an already-trained reviewer to reduce collapsed judgments, without retraining or new expert labels. The abstract gives no release date, license terms or code location beyond calling it open-source.

How solid is it

The evidence comes from a controlled experiment: a single base reviewer fine-tuned on six years of real ICLR reviews, then four successor models trained on ICLR 2024 data under systematically varied mixtures of real and synthetic reviews. That design lets the researchers isolate the effect of the synthetic share rather than just observing that AI was somewhere in the loop, which supports the claim that adding synthetic reviews specifically drives the compression. No numeric effect sizes are given in the abstract for either the collapse or the mitigation, so the magnitude of both remains unverified from this text.

Risks and caveats

The abstract names no individual authors or institutions, so provenance beyond the paper itself cannot be confirmed here. No percentage or other magnitude is given for how far rating distributions compressed or diversity fell, and no quantitative evaluation of TrustReviewer's effectiveness is reported, so how much the mitigation actually helps cannot be checked from this text. The experiment also covers one step of the feedback loop, successor models trained once on ICLR 2024 data with varied AI review shares; how the effect compounds over further recursive generations is not addressed here.

“Our study shows that introducing synthetic reviews compresses rating distributions and reduces both same-paper and corpus-level semantic diversity.”

— the paper's abstract