Glonce

Nvidia's SoL-Pi system cuts coding agent token usage nearly in half by optimizing the harness

researchThe Glonce editorsThe Decoder🔥75Glonce rating75Impact70Novelty68Relevance100Surprise52Players78Credibility82
Nvidia's SoL-Pi system cuts coding agent token usage nearly in half by optimizing the harness

Nvidia researchers have described a system called SoL-Pi that automatically optimizes the harness, the control layer sitting between an AI model and its environment, used by coding agents such as Codex, Claude Code and OpenClaw. Instead of cutting costs by shrinking the model or speeding up inference, as most efficiency work does, SoL-Pi targets how the harness manages tool calls, context, verification and abort logic, areas that are normally hand-tuned by engineers reading through execution traces.

SoL-Pi automates that process: a research agent watches another agent's traces, proposes changes to the harness, and tests candidates in prepared environments, keeping only those that pass capability and efficiency checks. The authors describe the approach as drawing on recursive self-improvement. To find improvements, the system searched across 535 executable environments and explored 152 directions, using 495 tasks derived from GitHub issue-pull-request pairs plus 40 synthetic test cases. The search generated more than 3,000 runs and over 60,000 agent-environment interactions. Because earlier work found that automatically optimized harnesses tend to overfit to their training tasks, the researchers walled off their EdgeBench test benchmark from the search entirely: of its 51 public tasks, 11 were used for one-time validation and the remaining 40 were reserved purely for final evaluation, never feeding back into the search.

The search produced four mechanisms. Action Fusion merges two consecutive steps, such as a code edit and a test run, into one, eliminating a model call. Online Context Compact trims accumulated context after each planning step without losing important information. ObservationPack archives long tool outputs and inserts a short summary on later steps instead of resending the full text. The Evidence-Preserving Reducer routes large error and test logs to a cheaper model that distills the key findings, with an automatic verification step catching critical details that might otherwise be lost.

On EdgeBench's 51 public tasks, SoL-Pi performed roughly on par with the original Pi harness. The most efficient configuration, combining all four mechanisms, used 49 percent fewer tokens while reaching 93.7 percent of Pi's score. A configuration that instead picked only the single strongest mechanism beat Pi's score by 5.3 percent while still cutting tokens; across the two variants, token usage dropped by 44.7 to 49 percent. In dollar terms, the authors estimate savings of $8.75 to $13.50 per hour compared with native Codex and Claude Code harnesses, and $4.36 to $5.71 per hour compared with Pi, based on current API prices. The system was built using only GPT-5.6 Sol, then applied unmodified to Opus 5, where it retained 94.3 percent of Pi's performance with similar savings, though the researchers note its mechanisms triggered less often and less aggressively there, which they attribute to the harness having been optimized solely on GPT-5.6 Sol trajectories.

Results were more mixed on other benchmarks. On 63 CPU tasks from Terminal-Bench 4, SoL-Pi solved only 15 tasks while Codex and Pi each solved 18, though its total costs still came in about a quarter lower than Pi's. On formally verified Lean 4 problems from the 2026 Math Olympiad, SoL-Pi solved three of six problems at the lowest cost per solved problem. In a kernel optimization experiment, a swarm of 20 SoL-Pi workers cut costs by 26.8 percent compared with a comparable Pi swarm, with total costs in one test run dropping from $1,339 to $894. The authors note the efficiency gains carry a trade-off, since shorter context can reduce prompt cache reuse. Looking ahead, they suggest pretraining a harness across many tasks and then using that lean harness to cheaply search for its successor, but they describe this recursive efficiency improvement as a vision rather than a finding of the current study.

The piece situates SoL-Pi against a broader cost problem in agentic AI. An August test by tooling company Composio running Deepseek V4 Flash across four agent frameworks, including Claude Code and the Pi-based Oh My Pi, found the cost per solved task varied by nearly 3x even with the same model doing the work. OpenRouter analyst Peter Walker has reported that agentic token usage has grown 14x since February 2026, with nearly 70 percent of that growth coming from cached prompts. Context compression of the kind SoL-Pi uses is not free of side effects either: one cited study found compression preserves only 17 percent of user instructions on average. Codex developer Eric Provencher has separately warned that using more than two sub-agents almost always burns tokens without improving quality, since they spend most of their time checking each other's work.

Key facts

  • SoL-Pi's most token-efficient configuration cuts token usage by 49 percent on the EdgeBench benchmark while reaching 93.7 percent of the original Pi harness's score; a performance-first configuration instead beats Pi's score by 5.3 percent while still saving tokens.
  • The search that produced SoL-Pi spanned 535 executable environments and 152 explored directions, using 495 GitHub issue-pull-request tasks and 40 synthetic cases, and generated more than 3,000 runs and over 60,000 agent-environment interactions.
  • Estimated hourly savings run $8.75 to $13.50 compared with native Codex and Claude Code harnesses, and $4.36 to $5.71 compared with the Pi harness, based on current API prices.
  • Applied unmodified to Opus 5 despite being built only on GPT-5.6 Sol, SoL-Pi retained 94.3 percent of Pi's performance with similar savings, though its mechanisms fired less often and less aggressively.
  • On 63 CPU tasks from Terminal-Bench 4, SoL-Pi solved only 15 tasks versus 18 each for Codex and Pi, though its total cost came in about a quarter lower than Pi's.

Why it matters

As coding agents run longer without supervision, their token consumption balloons through chains of reasoning, tool calls and feedback loops. Most efficiency efforts so far have targeted the model itself, through faster inference, quantization or cheaper models. SoL-Pi instead targets the harness, the control layer that governs how an agent sees state, takes actions and processes feedback, showing that a large share of agentic cost can be squeezed out without touching the underlying model at all.

Who it affects

The work matters most to teams and companies running coding agents such as Codex, Claude Code, OpenClaw or Pi-based systems at scale, where API costs scale directly with token usage. It is also relevant to anyone building or tuning agent harnesses, since the four mechanisms SoL-Pi discovered (Action Fusion, Online Context Compact, ObservationPack and the Evidence-Preserving Reducer) target generic inefficiencies rather than anything specific to one model.

How to use it

SoL-Pi is presented as a research system rather than a shipped product; the source gives no release date, availability timeline, or indication of whether the code or paper has been made public. For now the practical takeaway is the four mechanisms themselves and the demonstrated cost ranges, which teams tuning their own harnesses could use as a reference point.

How solid is it

The study comes from Nvidia researchers and includes an unusually careful evaluation design: the EdgeBench benchmark used for final scoring was walled off from the search process, with 40 of its 51 tasks held out purely for evaluation, precisely to guard against the overfitting that earlier automatically optimized harnesses showed. The system was also tested by transferring it unmodified from GPT-5.6 Sol to Opus 5, on Terminal-Bench 4 CPU tasks, on Lean 4 problems from IMO 2026, and in a 20-worker kernel optimization swarm, giving a broader picture than a single benchmark would.

Risks and caveats

Results are not uniformly positive: on Terminal-Bench 4's 63 CPU tasks, SoL-Pi solved fewer tasks (15) than Codex or Pi (18 each), even though its total cost was lower. The efficiency mechanisms transferred imperfectly to a different model (Opus 5), firing less aggressively than on the model they were built on. The authors themselves frame the idea of a fully recursive, self-pretraining harness as a vision rather than something demonstrated in this study. More broadly, the piece notes that context compression techniques like the ones SoL-Pi uses can have side effects, citing a study where compression preserved only 17 percent of user instructions on average, and that adding sub-agents beyond a small number tends to burn tokens without improving quality.