TokenEconomizer plugin cuts AI coding agent costs 18.4% to 30.9%

TokenEconomizer plugin cuts AI coding agent costs 18.4% to 30.9%

Developers tend to use AI coding agents in the same codebases day after day, yet agents often begin every session from scratch. They rediscover familiar facts, load tools they never use and repeat approaches that already failed. Every interaction with the model burns tokens: in each session an agent sends instructions, tool descriptions and session history. Most cost-cutting work targets single tasks, but the Microsoft Research post argues that repeated work in one repository is a chance to learn.

The paper is titled "TokenEconomizer: Experience-Driven Black-Box Cost Control for Coding Agents". TokenEconomizer is an open-source plugin from UC Berkeley, Microsoft Research and Microsoft Security. It turns completed sessions into guidance for later ones, and it does this without changing the agent or its underlying model, so it can sit alongside agents developers already use.

The team first profiled Claude Code on 60 SWE-Bench tasks, a benchmark of real bug fixes in open-source Python projects, running each repository's tasks in sequence. Three kinds of waste kept recurring. First, rediscovery: once a repository had at least five completed tasks, 30% of tool calls repeated an action from an earlier task in that repository, while only 7% matched an action in another repository's history. Second, unused tools: the agent received 23 tool descriptions on every turn, but 16 were never called. Third, getting stuck: the 17% of tasks that failed accounted for 32% of the spending. The authors also tested three popular open-source cost-saving plugins, each built around a fixed strategy. Each lowered costs in some repositories but not others, which the post takes as a sign that cost control has to adapt to the repository.

After each session TokenEconomizer reads the session log and updates what it knows about the repository. It then acts through three mechanisms, using only interfaces the agent already provides. Micro-skills are one-line repository-specific notes; the plugin spots recurring patterns, proposes a note, adds it to the agent's starting context only after several sessions have confirmed the pattern, and retires it if later evidence contradicts it. Tool pruning removes tools that repository history suggests the agent is unlikely to use, always keeping the core tools for running commands and editing files; removed tools can be restored if needed. Failure-loop intervention watches for an agent that keeps hitting the same failure. A judge model decides whether the agent is truly stuck or still making progress. If stuck, the agent gets a short note suggesting a different approach. In the open-source plugin, if the agent is still stuck after the note, TokenEconomizer alerts the developer rather than stopping the work itself. The controller is built to revise its decisions as new evidence arrives, and the reported savings include the cost of TokenEconomizer's own model calls.

For evaluation, the team used 277 SWE-Bench tasks from six repositories and five AI coding agent and model combinations. For each repository the plugin started with no prior session history. It lowered average cost by 18.4% to 30.9% and saved the most in all 30 repository-combination pairs without performance loss. Lowering the model's reasoning effort saved similar or larger amounts but caused substantial performance degradation. Costs fell in every repository tested, and task success rates remained broadly stable.

Savings grow with experience. Early tasks can cost more while the plugin learns, but cumulative costs fell below the default in every repository by the sixth task. Each mechanism saves money on its own, and combining all three gives the largest reduction. The savings also carried to sequential web development, with costs 16.0% to 21.2% lower on Web-Bench. On security auditing, where every task is in a new repository, costs were 6.4% to 19.6% lower.

The post closes by describing the plugin as designed to run alongside existing coding agents, needing no API key of its own, and keeping what it learns on the developer's machine. The authors invite the community to use it as a starting point for agents that become more resourceful with experience.

Key facts

  • TokenEconomizer is an open-source plugin from UC Berkeley, Microsoft Research and Microsoft Security that learns from past sessions in a repository without modifying the agent or its model.
  • Across 277 SWE-Bench tasks from six repositories and five agent and model combinations, it lowered average cost by 18.4% to 30.9%, with success rates broadly stable.
  • Profiling Claude Code on 60 tasks found 30% of tool calls repeated earlier actions in the same repository, 16 of 23 tool descriptions were never called, and the 17% of failed tasks took 32% of spending.
  • Three mechanisms do the work: micro-skills (one-line repository notes), tool pruning, and failure-loop intervention using a judge model.
  • Savings were also 16.0% to 21.2% on Web-Bench and 6.4% to 19.6% on security auditing, where every task is in a new repository.

Why it matters

Token spend is a running cost of using coding agents, and most cost-saving effort looks at one task at a time. This work targets the repetition across tasks in one repository. The profiling numbers show where the waste sits: 30% of tool calls repeated an earlier action in the same repository, 16 of 23 tool descriptions sent every turn were never called, and failed tasks (17%) consumed 32% of spending. The post also reports that simply lowering the model's reasoning effort saved similar or larger amounts but substantially degraded performance, whereas TokenEconomizer kept success broadly stable.

Who it affects

Developers who run AI coding agents repeatedly in the same codebases, and researchers working on agent efficiency. The plugin is meant to work alongside the coding agents developers already rely on, since it does not modify the agent or its model. The security-auditing result suggests some benefit even where every task is in a new repository, though the saving there is smaller.

How to use it

The post describes an open-source plugin designed to run alongside existing coding agents. It requires no API key of its own and keeps what it learns on the developer's machine. It acts only through interfaces the agent already provides. In practice, the more sessions in a repository, the more it has to work with: early tasks can cost more while it learns. No repository link, license or installation instructions appear in the text.

How solid is it

The source is a first-party Microsoft Research blog post describing the team's own paper, so the results are the authors' own. The evaluation covers 277 SWE-Bench tasks across six repositories and five agent and model combinations, with TokenEconomizer starting from no history in each repository. The reported savings include the cost of the plugin's own model calls. The post says it saved the most in all 30 repository-combination pairs without performance loss, and each of the three mechanisms contributes savings on its own. The size of the change in success rate is described only as broadly stable.

Risks and caveats

Early tasks can cost more while the plugin learns; cumulative costs fell below the default by the sixth task in every repository tested. Results come from SWE-Bench (Python bug fixes), plus Web-Bench and security auditing, and the savings vary by setting: 18.4% to 30.9% on SWE-Bench, 16.0% to 21.2% on Web-Bench and 6.4% to 19.6% on security auditing. Failure-loop intervention relies on a judge model to decide whether an agent is stuck, and in the open-source plugin a still-stuck agent leads to an alert to the developer, not an automatic stop. Tool pruning removes tools based on history, though removed tools can be restored if needed. The size of the task-success change is not quantified beyond "broadly stable".

“effective cost control must adapt to the repository, rather than apply the same strategy everywhere”

— Microsoft Research blog post on TokenEconomizer