Glonce

Retracted climate cost paper was driven by one bad Uzbekistan data point

researchThe Glonce editorsHacker News🔥46Glonce rating46Impact55Novelty50Relevance20Surprise65Players45Credibility92

A high profile paper by Kotz et al., published last year, claimed climate change would cost the world economy about 300% of what prior estimates had found, a figure that drew heavy attention in Europe and was used by central banks to run climate stress tests. Economist Sol Hsiang, running basic robustness checks across several papers by dropping one country at a time and rerunning the models, found that Kotz et al.'s results depended almost entirely on economic data for Uzbekistan covering 1995 to 1999. Remove that one country and the extreme numbers fall apart; the underlying costs are still large, but nowhere near what made headlines. A news article summarizing the story put a number on the swing: with Uzbekistan included, the paper projected a 62 percent decline in global economic output by 2100 under unabated emissions; excluding it, the projected decline was 23 percent, a near threefold difference from one data point out of a global dataset. Hsiang traced the fault to quality control: the paper's research assistants had transcribed numbers from a PDF document, and those transcribed Uzbekistan figures turned out to be implausible. A scatterplot in the original paper's companion data-validation paper was meant to catch exactly this kind of issue, but its axes had been cropped so the outliers, Uzbekistan included, did not visibly stand out; Hsiang's team reproduced the plot uncropped for their comment. The authors' own retraction note, quoted in full in the source, states that correcting the Uzbekistan data for 1995-1999, controlling for data source transitions and higher order trends in that data, and accounting for spatial auto-correlation changed the estimates enough that a correction was insufficient: the uncertainty range for mid-century climate damages widened from 11-29% to 6-31%, and the probability that damages diverge across emission scenarios by 2050 fell from 99% to 90%. All authors agreed to the retraction. A revised version of the paper, not yet peer reviewed, is posted publicly on Zenodo with its data and methods open, and the authors say they intend to resubmit it for peer review. A forthcoming comment in Nature by Hsiang and coauthors documents the episode. The retraction note thanks Thomas Bearpark, Dylan Hogan, Solomon Hsiang and Christof Schötz for flagging the issue. Commentators on the story, including Jonathan Falk, pointed to the episode as an argument for open data and methods: without access to the original dataset, catching the Uzbekistan problem would have been far harder.

Key facts

  • Kotz et al.'s paper claimed climate change would cost 300% of what prior estimates found, and was widely covered and used by central banks for climate stress tests.
  • Sol Hsiang's team could not reproduce the result; dropping Uzbekistan alone from the dataset collapsed the extreme cost estimate, though large underlying costs remained.
  • A news summary put the swing at a 62 percent projected decline in 2100 global output with Uzbekistan included versus 23 percent excluding it.
  • The Uzbekistan data for 1995-1999 traced back to numbers transcribed from a PDF by research assistants, later found implausible; a validation scatterplot had its axes cropped so the outliers were not visible.
  • The authors retracted the paper rather than issue a correction, after the fixes changed the mid-century uncertainty range from 11-29% to 6-31% and cut the probability of scenario-divergent damages by 2050 from 99% to 90%; a revised, not-yet-peer-reviewed version is on Zenodo.

Why it matters

The story is a case study in how a single flawed data point, if unchecked, can inflate a widely cited scientific estimate by nearly a factor of three and travel from a spreadsheet into headlines, European policy discussion and central bank stress tests before anyone catches it. It also shows a robustness check working exactly as intended: Hsiang's team found the problem simply by running the standard practice of dropping one country at a time and rerunning the model across several papers, not by targeting Kotz et al. specifically.

Who it affects

The paper's own authors, who retracted their work and are preparing a revised submission; central banks that had used the original estimates to run climate stress tests; and the broader climate economics field, which now has a concrete example of how much a single country's data quality can swing a global damage estimate.

How to use it

Not applicable: this is a research retraction and methodology story, not a product, service or purchasable tool.

How solid is it

The account comes directly from Sol Hsiang, a coauthor of the forthcoming Nature comment, and reproduces the retraction note in full; the note itself states all authors agreed to the retraction, and a revised dataset and methodology are posted openly on Zenodo, though that revision has not yet been peer reviewed.

Risks and caveats

The forthcoming Nature comment was not yet published at the time of writing, so its full analysis is not independently checked here; the revised version of the Kotz et al. paper is likewise still pending peer review. The source does not give the paper's original title, its original publishing journal, or an absolute dollar figure for the original cost estimate, only the relative claim that it was 300% of prior estimates.

“We couldn't reproduce their findings and realized that it was all driven by weird data from Uzbekistan.”

— Sol Hsiang