Micro benchmarks should run about 300ms, one blog post argues

A blog post titled "Benchmark In Milliseconds" asks how long a micro benchmark should run. Its answer is a personal rule of thumb: tweak the input size until the benchmark takes about 300ms.

The post gives four reasons. First, milliseconds are integers ranging from 1 to 999, which is enough precision to notice even a small improvement and easy to scan visually. There is no need for different units or floating points; the post contrasts 1.31s with 239ms.

Second, anything faster than, say, 10ms risks being skewed by fixed costs such as interpreter startup. Hundreds of milliseconds is an eternity for a computer, usually enough to make one-off overheads irrelevant without fancier (= less robust) techniques to account for them explicitly.

Third, for a human, hundreds of milliseconds is fast but noticeable. Pushing numbers into the human-perceptible range lets the author use an intuitive sense of time and speed rather than relying exclusively on numeracy. The post adds that it is plain fun to see a previously lagging CLI command become "instant" as a result of optimization work.

Fourth, anything longer than a second makes iterating on the benchmark slower than it needs to be. Running a benchmark 10 times in a row to eyeball variance should be fast.

The post closes by stating its underlying assumption: the purpose of benchmarking is not so much a precise measurement of performance as giving the author enough intuition to make a correct decision.

Key facts

The rule of thumb: tweak the input size until a micro benchmark takes about 300ms.
Whole milliseconds (1 to 999) are precise enough to spot small improvements and avoid mixed units or decimals, as in 1.31s versus 239ms.
Runs faster than, say, 10ms risk being skewed by fixed costs like interpreter startup; hundreds of milliseconds usually make one-off overheads irrelevant.
Anything longer than a second slows iteration, for example running the benchmark 10 times in a row to eyeball variance.
The stated assumption: benchmarking is meant to give enough intuition for a correct decision, not a precise measurement.

Why it matters

Choosing how long a benchmark runs is a small decision that quietly shapes the result. Too short and fixed costs distort the number; too long and every experiment drags. The post offers one simple target, about 300ms, and ties it to both machine behaviour and human perception. Its framing is that benchmarking serves decisions, so intuition matters more than a precise figure.

Who it affects

Anyone who writes micro benchmarks while optimizing code. The post's examples point to programs with startup costs, such as interpreter startup, and to CLI commands whose speed a person can feel.

How to use it

Adjust the input size, not the code under test, until a run takes about 300ms. Read the result as an integer number of milliseconds. Then repeat the run, for example 10 times in a row, to eyeball the variance. Keep each run under a second so this loop stays quick.

How solid is it

This is a personal rule of thumb, presented as such with the words "about 300ms". The 10ms threshold is illustrative ("say, 10ms"), not a precise cutoff. The reasoning is explained in the post's own terms.

Risks and caveats

No measurements, benchmark results or empirical evidence are cited for any of the four reasons. The 300ms figure is not a standard or a universal recommendation. The post itself says the goal is intuition for a decision, so it does not claim to deliver precise performance measurement.

“Hundreds of milliseconds is an eternity for a computer, usually enough to make one-off overheads irrelevant without using fancier (= less robust) techniques to explicitly account for them.”

— "Benchmark In Milliseconds" blog post