Spillbench finds register spill counts poorly predict runtime

An independent benchmark project, spillbench, tested a common assumption in compiler performance folklore: that starving a program of CPU registers slows it down roughly in proportion to how many spill instructions (extra stack loads and stores) the compiler is forced to generate. The authors compiled nine small C kernels, ranging from SHA-256 and SipHash to a FIR filter, LZ77, Quicksort and a double-precision matmul, with gcc 15.2.0 at -O2 on an Intel Xeon E-2236. Using the gcc -ffixed- flag, they progressively removed registers from the compiler's allocator pool, one at a time, sweeping general-purpose (GP) registers from 15 allocatable down to 5, and XMM (floating-point) registers from 16 down to 4. The ABI registers used for argument passing were never touched. At each budget they disassembled the hot function to count static stack-spill instructions (a proxy, since hardware performance counters were disabled on the test machine) and timed the workload with a Rust harness, taking the minimum of 15 runs on one pinned core. Every build passed its known-answer test and produced a bit-identical result at every budget, so only the generated code changed, never the output.

At the tightest register budget, 8 of the 9 kernels slowed down by 14 to 76%; SipHash-2-4 barely moved. Across all 105 (kernel, budget) measurement points, the Pearson correlation between added spill instructions and slowdown was only r = 0.55, and the cost of a single added spill varied by about 30 times between kernels, from roughly 0%/spill for SipHash to 2.2%/spill for the FIR filter. Integer matmul reached +42% slowdown with just 22 added spills, while LZ77 needed 76 added spills to reach +19%, and SipHash added 23 spills for no measurable slowdown (a small -2% shift, within noise). The authors' explanation is that the stack sits in L1 cache, so a spill off the critical dependency path is nearly free, while a spill inside a tight recurrence, such as an ARX chain or an accumulator update, stalls on reload latency.

The slowdown was also specific to which register file a kernel actually depends on. Reserving GP registers left the double-precision matmul flat (131.9 to 131.1 ms across a 0-to-10-register sweep), while reserving XMM registers on the same kernel cost +34%. SHA-256 was messier: reserving XMM cost only +5.6% while reserving GP cost +33%, yet the XMM reservation was not inert, since SHA-256's static spill count still rose from 8 to 65. The authors trace this to gcc auto-vectorizing part of SHA-256's message-schedule computation with SSE at -O2, so squeezing XMM registers pushes that work onto the stack even though it barely touches the critical path.

The post closes with an explicitly labeled extrapolation, not a measurement: reading the cost of x86-32-era register scarcity (6 to 7 usable GP registers instead of 15) off the same curves gives about 15% slowdown for a typical kernel and about 30% for the most register-hungry integer code, which the authors say makes the commonly cited "about 30%" figure a fair description of the hungry tail but roughly double the average case. They attribute the difference partly to out-of-order execution, which was absent on 486-class and original Pentium hardware and is what turns a modern spill from a pipeline stall into an often-free operation.

Key facts

Reserving registers with gcc -ffixed- slowed 8 of 9 tested C kernels by 14-76% at the tightest budget; SipHash-2-4 was the exception and barely moved.
Across 105 (kernel, budget) data points, added spill instructions correlated only weakly with slowdown (Pearson r = 0.55), and the cost per spill ranged about 30x between kernels, from roughly 0%/spill for SipHash to 2.2%/spill for a FIR filter.
The slowdown tracks the specific register file a kernel depends on: reserving GP registers left a double-precision matmul flat (131.9 to 131.1 ms) while reserving XMM cost it +34%; for SHA-256 the pattern reversed, with GP reservation costing +33% against +5.6% for XMM.
SHA-256's static stack-spill count still jumped from 8 to 65 under XMM reservation despite the small runtime hit, because gcc auto-vectorizes part of its message schedule with SSE at -O2, so the register file a kernel 'uses' is not a clean integer-vs-float split.
The authors explicitly label their x86-32-era extrapolation (about 15% typical, about 30% for the hungriest kernels) as back-of-envelope, not measured, and credit out-of-order execution with making modern spills far cheaper than they would have been on 486 or original Pentium hardware.

Why it matters

Compiler and performance folklore treats register pressure and spill counts as a reliable stand-in for runtime cost: fewer registers, more spills, slower code, roughly in proportion. This benchmark tests that assumption directly rather than assuming it, and finds the relationship is real in direction (removing registers does slow code down, on average) but far too noisy in magnitude to use spill counts alone as a performance signal. A correlation of r = 0.55 with a 30x spread in per-spill cost means the same static spill count can mean almost nothing or a lot, depending on whether the spilled value sits on a hot recurrence or off to the side.

Who it affects

Compiler engineers and anyone tuning register allocation heuristics, performance engineers who use static spill counts as a proxy metric when profiling counters are unavailable, and developers writing tight loops in register-starved contexts, such as inline assembly, embedded targets, or code that shares registers across inlined functions. It also speaks to anyone who cites the folk figure that 32-bit x86's smaller register file cost about 30% versus 64-bit, since the study argues that number describes only the register-hungry tail of programs.

How to use it

The benchmark and its full dataset are public at github.com/rjpower/spillbench, including the nine kernel sources, the Rust timing harness, the figure-generation script, and the raw results.json behind every reported number. The practical takeaway for anyone tuning code under register pressure: do not treat a rising static spill count alone as proof of a runtime regression, check whether the spilled values sit on a tight recurrence (accumulators, hash-chain state) rather than being read once, since that is what determines whether a spill is nearly free or seriously costly on modern out-of-order hardware.

How solid is it

The measurement is narrow by the authors' own account: one machine (an Intel Xeon E-2236), one compiler (gcc 15.2.0), one optimization level (-O2), and nine small, L1-resident microbenchmark kernels. Hardware performance counters were unavailable in the test environment, so the spill metric is a static count from disassembly rather than a dynamic spill/reload count, and wall-clock timing (minimum of 15 runs) carries roughly 1% measurement noise even after that averaging. Every build passed a known-answer test with bit-identical output at every register budget, which rules out the reservation silently changing what the code computes. The historical x86-32 section is explicitly flagged by the authors as extrapolation, not a measurement on period hardware.

Risks and caveats

The authors list clang/LLVM as untested and merely expected to produce different curves, since it uses a different register allocator. The reservation order was fixed (registers given up in one specific sequence) and a different order, or one that targets specific hot values, could shift the results. The kernels are small enough to stay resident in L1 cache throughout, so a program whose spills miss cache would likely show a steeper cost per spill than measured here. No author names or publication date appear in the source post, and the historical x86-32-era cost figures are the authors' own back-of-envelope extrapolation from this Xeon's data rather than a measurement taken on period hardware.

“We think the explanation is that the stack lives in L1, so a spill that is not on the critical dependency path is nearly free, while a spill inside a tight recurrence (the ARX chain, the accumulator update) serializes on the reload latency.”

— spillbench authors