Glonce

Peer review buckles as scientific publishing grows 5.6% a year

researchThe Glonce editorsArs Technica🔥58Glonce rating58Impact60Novelty45Relevance75Surprise50Players25Credibility75
Peer review buckles as scientific publishing grows 5.6% a year

Jason Semprini, a health economist at Des Moines University, studied whether policies mandating the HPV vaccine for elementary school students actually reduce cervical cancer rates in a population. His finding ran counter to the intuitive expectation: mandates don't do much to lower the overall rate, likely because some people find ways to avoid a vaccine they are required to take. He submitted the manuscript for peer review, the standard process where other researchers in the field assess a paper's validity, usually anonymously and as unpaid volunteer work. In Semprini's case the single assigned reviewer misread the study, mistakenly believing Semprini was questioning whether the HPV vaccine itself prevents cervical cancer rather than evaluating a mandate policy meant to raise vaccination rates. Papers are normally reviewed by two or three experts, so a misreading by one is usually caught by the others; here there was only one reviewer, and the paper was rejected. Semprini called the situation frustrating: 'There's a very big difference there,' he said, describing the gap between what he actually studied and what the reviewer thought he studied. Researchers describe this kind of breakdown as increasingly common. The number of papers indexed in the databases Scopus and Web of Science has been growing exponentially, at 5.6 percent a year, and reviewer capacity has not kept pace. By one estimate, researchers worldwide collectively spend 15,000 years of work on peer review annually; if that labor were paid, the share done in the US alone would cost $1.5 billion. As unpaid review demands grow against a shrinking pool of willing volunteers, cases like Semprini's, a paper rejected on the strength of one reviewer's misunderstanding, are what researchers point to as evidence the system is faltering.

Key facts

  • A single reviewer misread health economist Jason Semprini's manuscript on HPV vaccine mandates, mistaking a policy-effectiveness study for a claim about the vaccine itself, and the paper was rejected as a result.
  • Papers are normally assigned two or three reviewers, which usually catches one reviewer's misunderstanding; Semprini's paper had only one.
  • The number of papers indexed in Scopus and Web of Science has been growing exponentially, at 5.6 percent per year.
  • One estimate puts researchers' collective peer-review workload worldwide at 15,000 years of labor annually, which if paid would cost $1.5 billion for the US share alone.
  • Semprini's own research found that HPV vaccine mandates for elementary school students don't do much to reduce a population's overall cervical cancer rate, a counterintuitive result he attributes partly to people finding ways around a mandate.

Why it matters

Peer review is the check that decides which research gets published, and it runs almost entirely on volunteer labor. With papers accumulating at 5.6 percent a year in major databases, the reviewer pool is being stretched thinner each year without any equivalent increase in paid capacity. Semprini's case shows the practical failure mode: when a paper draws only one reviewer instead of the usual two or three, a single misunderstanding is enough to sink it, with no second opinion to catch the error.

Who it affects

Researchers submitting papers, like Semprini, who can lose work to a reviewer's misreading with no built-in safeguard; the volunteer reviewers themselves, whose unpaid time is estimated at 15,000 years of collective annual labor worldwide; and journals and editors trying to secure enough qualified reviewers per submission as the volume of manuscripts keeps climbing.

How to use it

For anyone submitting research, the case is a reminder that a single-reviewer assignment carries real risk of an outcome turning on one person's misreading rather than a consensus judgment, since the two-or-three-reviewer norm exists precisely to catch that kind of error. For those asked to review, the story puts a number on the true cost of the favor: a share of a workload estimated, in the aggregate, at 15,000 unpaid years a year, worth $1.5 billion in the US alone if it were compensated.

How solid is it

The central case is a named, checkable researcher describing his own experience and institution. The systemic figures, the 5.6 percent annual growth in Scopus and Web of Science and the 15,000-year, $1.5 billion workload estimate, are both attributed only to 'one estimate' without a named source, so their methodology and currency can't be verified from what's given here.

Risks and caveats

The account rests on one researcher's anecdote generalized into a systemic claim, backed by unsourced aggregate figures rather than a named study. No time period is given for the 5.6 percent growth rate, so it's unclear over what span that exponential trend was measured. The piece also does not name the journal that rejected Semprini's manuscript or date the submission and rejection.

“There's a very big difference there,”

— Jason Semprini, health economist at Des Moines University