Glonce

METR finds AI dramatically accelerating vulnerability reporting in 2026

researchThe Glonce editorsImport AI🔥65Glonce rating65Impact55Novelty55Relevance100Surprise40Players55Credibility80
METR finds AI dramatically accelerating vulnerability reporting in 2026

The newsletter Import AI, in issue 470, opens with an analysis from AI evaluation group METR on where artificial intelligence has actually sped up scientific and technical progress. METR looks at three areas: cybersecurity, mathematics research and the optimization of AI research itself. In cybersecurity, the rate of vulnerabilities reported has, in METR's words, dramatically accelerated in 2026 compared with 2025, both for specific projects such as cURL, OpenSSL, Firefox and Microsoft and for aggregate vulnerability databases, the US NVD and OSV. In mathematics, AI is contributing to more research output, arXiv submissions have doubled in some areas in under 12 months, but METR calls the value of that work hard to quantify and says it may be too early to tell whether the trend will hold; around the same period, several problems from prestigious lists have been solved, including the Jacobian conjecture from Smale's list, Problem 44 (the halving sieve) from Green's list and the sofic half of Green's Problem 100, though METR does not credit AI with solving them directly. In the third area, tested across seven benchmark problems, CIFAR-10, Hutter compression, Gurobi mixed-integer programming, MIPLIB, nanoGPT, Stockfish and the matrix-multiplication exponent, METR finds no measurable acceleration overall, though AI-attributable progress does show up on two of the seven, nanoGPT and CIFAR-10.

A second item covers SPADE (Self-Play in Adaptive Synthetic Executable Environments), a framework built by a team spanning nine universities: the University of Washington, Stanford University, Northeastern University, Carnegie Mellon University, MIT, the National University of Singapore, Seoul National University, the Stevens Institute of Technology and the University of Chicago. SPADE has one model alternate between two roles: an Environment Designer that writes executable training puzzles as Python code, such as simulated biology-lab problems, and a Reasoning Agent that tries to solve them. The Designer also attaches what the authors call a privileged hint, extra information that makes a puzzle easier to solve; the gap between the Agent's performance with and without that hint becomes the Designer's own reward signal. The authors tested three Qwen3 backbones, Qwen3-4B-Instruct-2507, Qwen3-8B and Qwen3-30B-A3B-Instruct-2507, each tuned with GRPO over 400 rollouts of 25 environments and then tested against benchmarks including AIME, GPQA, LCB and Reasoning Gym environments. The largest model benefited most: at the 30B-A3B scale, SPADE reached a suite average of 58.3 on game-style environments, up 8.1 from the untrained base model and up 5.3 from the best fixed-environment baseline, and the same approach improved every backbone tested on tool-use environments too, though without a comparable published number. Code and model checkpoints are published on GitHub under the name spade-rl. The authors themselves caution that the technique does not let a model bootstrap far beyond the imaginative range of whatever base model generates the environments in the first place.

A third item covers Hawkeye, an open-source framework from researchers at Harvard, Stanford, Together AI and Caltech, built to help coding agents write well-optimized GPU kernels for specific hardware with minimal human input. Hawkeye packages a taxonomy of unit tests, each pairing a human-written solution kernel with the profiling metric that verifies an optimization, so a coding agent can read the kernel as a worked example, call it directly, or reuse pieces of it. The researchers tested Hawkeye by porting PyTorch workloads into kernels across NVIDIA's Ampere, Hopper and Blackwell chips and AMD's MI350, in precisions from BF16 down to FP8, NVFP4 and MXFP4. On established workloads, where torch.compile normally dispatches to expert-tuned vendor libraries like cuBLAS, cuDNN and FlashAttention, Hawkeye matched or beat those libraries in both BF16 and low precision, including in formats PyTorch cannot run natively. On newer attention variants that torch.compile cannot fuse, Hawkeye reached an 18.9× geometric-mean speedup over expert-written Triton kernels from the Flash Linear Attention library; on Linear Attention specifically, it matched or beat that library across every architecture tested, 1.22× faster on Blackwell and at exact parity, 1.00×, on AMD's MI350. The researchers also report that giving Hawkeye more test-time compute produces the most performant kernels across every architecture they tried.

The issue closes with a personal essay from AI researcher Julian Togelius, who is making public a piece, titled 'Losing my religion,' that he wrote during what he calls a crisis of faith about AI research in 2025. Togelius writes that he sometimes wakes at 3 am, heart pounding, dreading a future where human talent, knowledge and even genius stop mattering, and that greater technological capability could leave people unable to make a difference and with little reason to keep understanding more, in his words, abundance bought at the price of redundance. Import AI's writer notes that Togelius is not alone, citing Turing Award winners Geoffrey Hinton and Yoshua Bengio as researchers who have redirected their careers toward public-policy advocacy over AI's impact, and mentions writing a related essay of their own the previous year, titled 'Technological Optimism and Appropriate Fear' (Import AI issue 431).

Key facts

  • METR finds AI-linked vulnerability reporting dramatically accelerated in 2026 versus 2025 across projects like cURL, OpenSSL, Firefox and Microsoft and databases like the US NVD and OSV, while mathematics research saw only minor acceleration and the optimization of AI research itself showed no measurable acceleration across seven benchmark areas.
  • A team spanning nine universities built SPADE, which has a Qwen3-30B-A3B model alternate between designing synthetic training puzzles and solving them, lifting its game-benchmark suite average to 58.3, up 8.1 from the untrained base model and up 5.3 from the strongest fixed-environment baseline.
  • A Harvard, Stanford, Together AI and Caltech framework called Hawkeye let coding agents match or beat expert-tuned kernel libraries like cuBLAS and cuDNN on established GPU workloads, and reach an 18.9× geometric-mean speedup over expert-written Triton kernels on newer attention variants that torch.compile cannot optimize.
  • AI researcher Julian Togelius published a 2025 essay, titled 'Losing my religion,' describing a crisis of faith over what AI research succeeding would mean for human purpose, joining Turing Award winners Geoffrey Hinton and Yoshua Bengio among researchers who have grappled publicly with AI's implications.
  • Neither SPADE nor Hawkeye claims unlimited self-improvement: SPADE's authors say it cannot bootstrap a model past the imaginative range of whatever base model designs its training environments, and Hawkeye's own Linear Attention result on AMD's MI350 chip is only a tie with the existing expert-written library, not a win.

Why it matters

Import AI frames all four items around AI's growing role in accelerating its own progress. On METR's finding, the newsletter's own theory is that acceleration arrives as a sudden, hard-to-explain phase change in a given skill, something it says has already happened with day-to-day coding in 2025 and cybersecurity in 2026, leaving open whether other fields will see the same jump. SPADE is framed as a step toward recursive self-improvement: a cheap way to generate large, varied training data by having a capable model design puzzles for another to solve, though the newsletter notes the authors' own caveat that it cannot let a model bootstrap far past the imagination of whichever base model designs the environments. Hawkeye is framed the same way from a different angle: with a small amount of human-curated, gold-label knowledge packaged as unit tests, AI systems can already match or exceed expert-optimized human work, and the newsletter's view is that humans writing more tools like it is how machines end up bootstrapping past human capability. Togelius's essay closes the loop by asking what any of that is for: he is described as one of several prominent researchers, including Turing Award winners Geoffrey Hinton and Yoshua Bengio, who have grappled publicly with what a world where human talent and understanding stop mattering would mean.

Who it affects

AI safety and evaluation researchers get a concrete data point for arguments about differential acceleration and self-improvement. The specific projects METR names, cURL, OpenSSL, Firefox and Microsoft, along with the operators of vulnerability databases like the US NVD and OSV, are the ones whose 2026-versus-2025 numbers the analysis is built on. Machine learning researchers training models with reinforcement learning get a public recipe and released checkpoints in SPADE for generating synthetic training environments instead of hand-building them, tested so far on Qwen3 models at 4B, 8B and 30B parameter scales. Anyone optimizing AI workloads on NVIDIA's Ampere, Hopper and Blackwell chips or AMD's MI350 gets an open-source tool in Hawkeye for automating kernel tuning. And the wider AI research community is Togelius's audience, including, per the newsletter, prominent figures like Geoffrey Hinton and Yoshua Bengio who have shifted toward public-policy advocacy on AI's impact.

How to use it

SPADE ships as open code and model checkpoints under the name spade-rl on GitHub. It was tested on Qwen3 backbones at three sizes, 4B, 8B and 30B-A3B parameters, tuned with GRPO over rollouts of self-generated environments, so teams with their own reinforcement-learning pipeline can plug it in directly. Hawkeye is also open source: it is applied by routing existing PyTorch workloads through its unit-test taxonomy to generate hardware-tuned kernels, and has been tested across NVIDIA's Ampere, Hopper and Blackwell chips and AMD's MI350 in BF16, FP8, NVFP4 and MXFP4 precision. METR's piece is a research note, not a tool, useful for calibrating expectations rather than building anything. Togelius's essay is a piece of writing published on his own blog, not software.

How solid is it

METR's comparison rests on checkable numbers: vulnerability counts from named projects and databases, arXiv submission counts, and a defined list of seven benchmark problem areas. But METR itself calls the mathematics finding hard to quantify and says it may be too early to know whether the trend holds, and for AI-research optimization it does offer a reason: the rate of increase in AI usage there is a lot less than in cybersecurity and mathematics. SPADE and Hawkeye are each backed by their own paper's benchmark results (SPADE on arXiv, Hawkeye on alphaxiv) rather than independent replication. SPADE's gains are measured against a fixed-environment baseline across three model sizes: at the largest, a suite average of 58.3, up 8.1 from the untrained base model and up 5.3 from the strongest fixed-environment baseline. Hawkeye's headline 18.9× speedup applies specifically to newer attention variants that torch.compile cannot fuse; on Linear Attention specifically the gain is a more modest 1.22× on Blackwell, and on AMD's MI350 it is exact parity, 1.00×, not an improvement. Togelius's piece is a personal essay, offered and read as testimony rather than as a data-backed claim.

Risks and caveats

METR is explicit that the mathematics finding may be too early to call a real trend, and the newsletter is careful not to credit AI with directly solving the prestigious math problems it lists, only that they were solved around the same period. SPADE's own authors caution that the method cannot bootstrap a model meaningfully beyond the imaginative range of whatever base model generates its training environments, an explicit ceiling on the self-improvement story. Hawkeye's results are not uniformly positive: on AMD's MI350, its Linear Attention result is parity with the existing expert-written library, not a win. And the whole issue is one newsletter's own first-person curation and analysis, its recurring 'why this matters' framing included, rather than an independent or peer-reviewed synthesis of the four pieces of research.

“I sometimes wake up at 3 am, heart pounding, from the dread of a future where human talent, knowledge, and even genius does not matter”

— Julian Togelius