Glonce

LLMs crack double-blind peer review anonymity, study finds

researchThe Glonce editorsarXiv🔥73Glonce rating73Impact72Novelty68Relevance95Surprise62Players22Credibility78

Double-blind peer review is described by the authors as the scientific community's primary defense against status and affiliation bias: it works only if an anonymized manuscript conveys scientific merit without revealing who wrote it. The authors note that authorship can often already be recovered through citation networks or stylistic markers, but they set out to test a harder case: whether large language models can identify authors using nothing but a paper's title and abstract. They ran the test on papers published after the models' training cutoff, so the models could not have simply memorized the paper, and gave each model a pool of five domain expert candidates to choose from per paper. The result was that LLMs collapsed anonymity more efficiently than humans did, with the models' belief concentrating onto a small subset of plausible authors from that five-candidate pool. The effect held up even after the authors excluded stylistic and bibliographic cues, the kind of surface clues (writing style, citation patterns) that previously explained deanonymization. That persistence, the authors argue, points to something deeper: stable patterns in how a researcher frames a problem and where their research focus lies appear to function as a kind of latent conceptual signature of authorship, one that survives even when the obvious tells are removed. The authors conclude that double-blind review is vulnerable to this kind of automated semantic inference, and that the research community needs to reevaluate how it maintains anonymity and fairness in review now that AI tools are part of the research ecosystem.

Key facts

  • LLMs identified likely authors of anonymized papers using only the title and abstract, more efficiently than humans did.
  • Each paper's candidate pool was restricted to five domain experts, and the models' guesses concentrated on a small subset of that pool.
  • Papers were published after the models' training cutoff, ruling out simple memorization of the specific paper.
  • The deanonymization held up even with stylistic and bibliographic cues excluded, pointing to conceptual framing itself as an authorship signature.
  • The authors call for a reevaluation of how double-blind review maintains anonymity and fairness in an AI-augmented research ecosystem.

Why it matters

Double-blind review exists specifically to keep a reviewer's judgment of a paper's merit separate from who wrote it, protecting against bias toward well-known names or prestigious institutions. This study argues that protection is weaker than assumed: an LLM reading only a title and abstract, with none of a manuscript's stylistic tells, can still narrow authorship down to a small set of likely candidates. If confirmed and reproduced more broadly, that undercuts a core assumption the peer review system runs on.

Who it affects

The finding concerns anyone who relies on double-blind review to be genuinely blind: authors submitting anonymized manuscripts, program committees and journal editors who administer the process, and the broader research community that treats blind review as a bias safeguard. The source text does not specify which academic field or venue the tested papers came from, so it is unclear how far the finding generalizes beyond the study's own sample.

How to use it

There is no product or tool here to adopt; the practical takeaway is for people who run or trust double-blind review processes. The authors frame their result as a reason to reevaluate anonymity and fairness practices in review now that AI tools can plausibly assist with, or be used for, deanonymizing submissions.

How solid is it

The claim rests on a controlled test: candidate pools fixed at five domain experts per paper, papers drawn from after the models' training cutoff to prevent memorization, and stylistic and bibliographic cues explicitly excluded in at least one condition. That said, the source text does not give the accuracy or success rate at which models correctly picked the true author, does not name which LLMs were tested, and gives no comparison figure for human accuracy versus model accuracy, only the qualitative claim that models were more efficient. Without those numbers, the size of the effect is hard to judge from the abstract alone.

Risks and caveats

Several details that would help assess the result are absent from the text: no author names or institutional affiliations for the study itself, no named models, no specified academic field or venue for the test papers, and no methodology beyond the title-and-abstract setup with five-candidate pools. The claim of LLMs being more efficient than humans is qualitative rather than backed by a stated percentage, so readers should treat the magnitude of the risk as unquantified pending the full paper.