Pi's minimalist harness beats Claude Code and Codex on cost
Earendil, the maker of the coding harness Pi, published a post arguing that minimalism, not added complexity, is what makes a coding agent harness cheap and effective. Pi ships with only 4 tools out of the box, and its system prompt plus tool definitions come in below 1,000 tokens combined, in contrast to an industry trend toward larger prompts, more orchestration layers and more built-in tooling.
The post leans on two external case studies rather than internal benchmarks. The first is Databricks' own study, "Benchmarking Coding Agents on Databricks' Multi-Million Line Codebase," built from tasks its engineering team regularly performs rather than public benchmarks it considers oversaturated. Databricks reported that the harness a model runs through, not just the model itself, changes cost and quality. When Pi was combined with Claude Opus 4.8 at xhigh thinking effort, it had the highest overall pass rate among the harnesses tested, at a lower cost than both Claude Code and Codex. Running the identical model at the identical thinking effort through different harnesses changed the cost per task by more than 2x in some cases while quality held steady, and Databricks found Pi sent about 3x less context per turn, keeping a tighter working set and finishing tasks in fewer runs.
The second case study is Shopify. Engineer David Cortés built an extension called pi-autoresearch directly inside Pi by simply asking Pi to create an extension for autoresearch, since Pi reads its own extension documentation. The extension runs an autonomous loop that tests changes, keeps improvements and discards regressions for as long as the target is measurable. Shopify reported unit tests running 300 times faster, React component mounting 20% faster, reduced build times across multiple projects, and improvements to pnpm performance.
Earendil frames this as evidence that the old argument for native, model-specific harnesses has weakened: frontier models now handle terminal-style coding environments well regardless of which harness calls them, and it cites Anthropic's own 80% cut to Claude Code's system prompt as a sign of the same shift. Earendil adds its own observation that running complex workflows on Haiku 4.5 was often pricier than on Sonnet 4.6 when code execution was involved, because the smaller model needed more turns to finish, and argues the same dynamic now plays out at the harness level: a stronger, pricier model paired with an efficient harness can end up cheaper than a weaker model paired with a bloated one. It also points to context discipline, keeping the prompt prefix stable unless the user asks for a change, as an advantage for local models, which typically have smaller context windows and slower prefill.
Key facts
- Pi ships with only 4 tools and a system prompt plus tool definitions totaling below 1,000 tokens.
- In Databricks' own benchmark on its multi-million-line codebase, Pi combined with Claude Opus 4.8 at xhigh had the highest pass rate at a lower cost than both Claude Code and Codex.
- Databricks found that running the same model through different harnesses changed cost per task by more than 2x in some cases at equal quality, and that Pi sent about 3x less context per turn.
- Shopify engineer David Cortés built a pi-autoresearch extension by asking Pi to create it, then used it to speed up unit tests by 300 times and React component mounting by 20%.
- Earendil cites Anthropic's 80% cut to Claude Code's system prompt as evidence that how a harness manages context now matters more than how native it is to a given model.
Why it matters
The post pushes back on an industry trend Earendil describes as making tools bigger and more complex now that AI has made code cheap to write, arguing that added orchestration and layers mainly add cost without improving outcomes. Its central claim, backed by Databricks' own benchmark, is that harness design is a separate variable from model choice and can swing both cost and pass rate on its own.
Who it affects
Engineering teams choosing or building coding-agent harnesses, organizations running their own internal benchmarks of coding agents rather than relying on public leaderboards, and users of local models, where Earendil says Pi's stable context prefix avoids the slow re-prefilling that smaller context windows make costly.
How to use it
Pi is designed to be extended rather than to ship every tool by default. Shopify's David Cortés built the pi-autoresearch extension simply by asking Pi to create an extension for autoresearch, since Pi can read its own extension documentation and build from there; Earendil describes this self-editability as the intended way to add capability only where it earns its keep. The source gives no pricing or access details for Pi.
How solid is it
The claims rest on two external, named sources rather than Earendil's own numbers: Databricks' published benchmarking study and Shopify's internal use of a self-built extension, both quoted directly in the post. That said, this is a vendor post by Pi's own maker, and it gives no dates for when either study or extension was produced, no absolute dollar costs (only the 2x and 3x multiples), no individual Databricks researchers, and no detail on what the benchmark tasks specifically involved beyond being drawn from engineers' regular work.
Risks and caveats
As a promotional post from the tool's own vendor, the framing is one-sided even where the cited data comes from third parties; readers cannot check the underlying Databricks or Shopify reports from this post alone. The cost comparisons are relative multiples rather than absolute figures, and the claim that native harness advantages have weakened is Earendil's own interpretation of the trend, not a finding either case study states directly.
“the cost per task differed significantly (more than 2x in some cases), while quality remained the same”
— Databricks' benchmarking study, as quoted in Earendil's post