Twice-Told Critiques paper: ICLR review overlap rose from 12.0% to 21.3%

A paper titled "Twice-Told Critiques" starts from a simple premise: peer review relies on several readers finding different problems. If reviewers of one paper raise the same points, the process adds less than it should. The authors measure critique overlap in conference and journal reviews.
At ICLR, measured overlap rose from 12.0% in 2026 to 21.3% in 2027, in reviews from the same stage of the review process. The paper also reports year-on-year rises at NeurIPS, ICML and the F1000 and PLOS journals. The summary gives no figures for those venues.
A second measure points the same way. A third ICLR review of a paper added 5.46 new problems to the other two in 2027, against 6.12 in 2026.
To bring LLMs into it, the authors apply MELD, a detector of LLM-generated text, to all reviews. Within a single paper, two reviews that MELD flags overlap more than two unflagged reviews. The gap is 3.0 points at ICLR in 2024-2025 and 4.5 points in 2026-2027. Across five journal sources it is 5.4 points.
The authors conclude that these results provide evidence that LLM-flagged reviews of one paper are less distinct from each other than unflagged reviews. They say the results suggest that the diversity of critiques among reviews of a single paper is falling.
Key facts
- At ICLR, measured critique overlap between reviews rose from 12.0% in 2026 to 21.3% in 2027, for reviews from the same stage of the review process.
- A third ICLR review of a paper added 5.46 new problems to the other two in 2027, against 6.12 in 2026.
- Two reviews of one paper that the MELD detector flags as LLM-generated overlap more than two unflagged reviews: by 3.0 points at ICLR in 2024-2025, 4.5 points in 2026-2027, and 5.4 points across five journal sources.
- Year-on-year rises in overlap are also reported at NeurIPS, ICML and the F1000 and PLOS journals.
- The authors say the results suggest, rather than prove, that critique diversity among reviews of one paper is falling.
Why it matters
Peer review is built on the idea that several readers will each catch different problems. If their critiques increasingly coincide, each extra reviewer contributes less. The ICLR figures show this in two ways: overlap climbing from 12.0% to 21.3%, and a third review adding 5.46 new problems in 2027 against 6.12 in 2026. The paper ties part of the trend to LLM-flagged reviews, which overlap more with each other than unflagged ones do.
Who it affects
Conference and journal organisers at ICLR, NeurIPS and ICML, and the F1000 and PLOS journals, where the paper reports rising overlap. Reviewers and authors are affected too, since the number of distinct problems a paper gets flagged depends on how different the reviews are.
How to use it
This is a measurement, not a tool. The paper offers a way to look at review quality: compare the critiques in different reviews of one paper, and compare detector-flagged reviews with unflagged ones. Programme chairs and editors can use the figures as a reference point when thinking about how many reviews a paper needs.
How solid is it
The source is a short abstract on Hugging Face Papers. The authors word their claim carefully: the results "provide evidence" that flagged reviews are less distinct from each other, and "suggest" that diversity is falling. The abstract does not define how critique overlap is computed or what the percentage is a share of. It does not say that LLM use caused the rise; the MELD comparison is between flagged and unflagged reviews within a paper. It gives no overlap figures for NeurIPS, ICML, F1000 or PLOS, no accuracy or false-positive rate for MELD, and no count of reviews or papers analysed.
Risks and caveats
The LLM link rests on a detector of LLM-generated text, and the abstract does not report how reliable MELD is. The flagged-versus-unflagged gap (3.0, 4.5 and 5.4 points) is a correlation within papers, and the authors only say the results suggest falling diversity. The rise in overlap also shows up as a time trend, so it should not be read as proof of any single cause. The abstract proposes no remedy or policy change.