OpenAI field report: AI coding agents speed research code, can't verify results

OpenAI and academic partners published a field report documenting eight case studies, mostly in biology, where research groups used AI coding agents such as Codex and Claude Code to modernize, optimize or rewrite aging research software. The report's central finding: agents speed up implementation, but they cannot reliably judge whether the resulting science is correct, and even when their code contains errors the systems often present it with full confidence.

The case studies span a range of ambition. GPT-5.5 modernized the build and installation process for cyvcf2, a Python library for reading genetic data. For MHCflurry, an immunology model that predicts which targets immune cells will recognize, Claude Code and Codex alternated between developer and reviewer roles while porting about 10,000 lines of code from TensorFlow to PyTorch; an earlier attempt to make that same port had failed in early 2025, which developer Sergey Feldman now attributes to the models available at the time rather than the coding tools themselves. The most ambitious rewrite, rustar-aligner, rebuilt the sequencing tool STAR, more than 20,000 lines of C and C++ that remain part of many research pipelines despite no longer being actively maintained, from scratch in Rust. Tested against 10,000 short sequencing reads from yeast cells, rustar-aligner matched STAR's output in 99.815 percent of single-end reads and 99.883 percent of paired-end reads, with neither tool mapping a read the other failed to map.

The largest speedups came from RustQC and HelixForge. RustQC, led by Philip Ewels, combined 15 separate quality-control tools into a single program and cut runtime on a large dataset from 15 hours and 34 minutes to 14 minutes and 54 seconds, a speedup of more than 60 times. HelixForge, a GPU version of a tool for generating synthetic genomic data, completed a full pipeline 59.6 times faster than the original BamSurgeon on a ten-million-base-pair section of one donor's genome, with the main compute step alone running 98.6 times faster. Smaller optimization passes also paid off: GPT-5.5 cut hifiasm's runtime on real human genome data by nearly 15 percent using a researcher-built training and validation test setup, and a GPT-5.2 pass followed by a second pass with a newer model reduced HI.SIM's runtime by about 31 percent without changing its output.

Not every rewrite was clean. The bayesm Rust rewrite ran between two and twenty times faster than the original, but early versions of two advanced methods contained errors that were hard to spot from the output alone. In one, the agent inverted a key control parameter, causing the program to use the reciprocal of the intended values; researchers only caught it after running a detailed calibration test against thousands of synthetic datasets with known results. A second method, HART, produced plausible-looking results while still containing needlessly expensive calculations and an incorrectly scaled correction factor. "With coding agents, it's quite easy to go fast; for now, to go far in science, there's still a need for expert guidance, understanding, taste, and care," writes cyvcf2 developer Brent Pedersen. Ewels never let the models judge the accuracy of their own work, building an independent test harness instead.

The authors estimate the stakes in dollar terms: if agents could resolve between one-quarter and one-half of all installation problems affecting research software, the saved research time across 100 packages would be worth between $600,000 and nearly $5 million; for NumPy alone, they estimate agents could save about 650 hours of maintenance work a year. But cheap rewrites carry their own risk of fragmenting communities and spreading already limited maintainer time thinner. Teams handled ownership differently: rustar-aligner moved to the scverse research consortium because STAR itself was unmaintained, while FastQC's author declined to replace the original tool with its Rust rewrite and instead folded the same improvements into the existing Java version, reaching the same threefold speedup without a rewrite.

The report's authors stress their findings are not from a representative study, and they see the bottleneck shifting away from coding itself toward validation, scientific review and maintenance responsibility. A similar pattern shows up outside research software: a METR study found that actual project maintainers would reject about half of the solutions the widely used SWE-bench Verified benchmark rates as passing, and the curl project shut down its bug bounty program after AI-generated vulnerability reports consumed maintainers' time without producing useful results. The field report is part of OpenAI's broader push into science, run by a dedicated team led by Kevin Weil, who expects 2026 to be for science what 2025 was for software engineering; in April, OpenAI introduced GPT-Rosalind, a model for life sciences research, and released a free Codex life sciences plugin connecting to more than 50 public databases and biology tools.

Key facts

Eight case studies, mostly in biology, used coding agents such as Codex and Claude Code to modernize, optimize or rewrite aging research software.
RustQC combined 15 quality-control tools and cut runtime from 15 hours 34 minutes to 14 minutes 54 seconds, over 60 times faster; HelixForge ran 59.6 times faster overall and 98.6 times faster in its main compute step.
rustar-aligner, a from-scratch Rust rebuild of the 20,000-line STAR aligner, matched the original's output in 99.815 percent of single-end and 99.883 percent of paired-end test reads.
The bayesm Rust rewrite ran 2 to 20 times faster but shipped with a bug that inverted a key control parameter, caught only through calibration against thousands of synthetic datasets.
Authors estimate agents could save $600,000 to nearly $5 million in research time across 100 packages, and about 650 hours a year for NumPy alone, if they resolve one-quarter to one-half of installation problems.

Why it matters

Much widely used research software started as supporting code for a single paper, written by small academic teams without time or budget for proper testing or maintenance; it stays fragile even as entire fields depend on it. The report shows AI coding agents can genuinely close that gap on the implementation side, in one case cutting a 15-hour run to under 15 minutes, but the effort saved on writing code reappears as effort spent verifying it. That reframes the promise of coding agents in science: speed was never the bottleneck, correctness was, and the report is OpenAI's evidence for its bet, voiced by science-team lead Kevin Weil, that 2026 will do for scientific software what 2025 did for software engineering generally.

Who it affects

Biology and life-science researchers who depend on tools like STAR, FastQC, NumPy and MHCflurry sit at the center of the report; the case studies were run by developers such as Brent Pedersen (cyvcf2), Philip Ewels (RustQC) and Sergey Feldman (the MHCflurry port). It also bears on open-source maintainers generally, given the article's own parallel to curl shutting down its bug bounty program after AI-generated reports wasted reviewer time, and on OpenAI's academic partners and science team as the effort scales toward more packages.

How to use it

This is a field report on method, not a product with a price. The recipe it documents: humans set the goals, success criteria and validation methods up front, including independent test harnesses and calibration datasets, while agents such as Codex, Claude Code, GPT-5.5 and GPT-5.2 handle the implementation. Crucially, none of the teams let the agents judge their own output; Philip Ewels built a separate test harness specifically because agents present wrong code as confidently as correct code.

How solid is it

The report's own authors caution that it is a retrospective look at completed projects based on participants' accounts, not a representative study, and the article does not name the academic partner institutions, give a publication venue or peer-review status for the field report, or say which year the cited April GPT-Rosalind launch falls in. The core caution about validation gaps is corroborated from outside the report: a METR study found that real project maintainers reject about half the SWE-bench Verified solutions the benchmark scores as passing.

Risks and caveats

The report's headline risk is that agents can be, in Philip Ewels' words, confidently wrong in ways that are easy to miss; the bayesm case shows an inverted control parameter and a separate calculation bug survived until researchers ran calibration against thousands of synthetic datasets, and the HART method looked plausible while still containing an incorrectly scaled correction factor. Beyond correctness, the report flags a maintenance risk: cheap, low-cost rewrites could fragment user communities and spread already-scarce maintainer time even thinner. The article also does not break down how much of the projected $600,000-to-$5-million saving, or the 650 NumPy hours, has actually been realized versus estimated.

“eloquent, convincing, and confidently wrong in ways that are easy to miss”

— Philip Ewels, who led the RustQC rewrite