Databricks shares how it cuts AI coding costs at scale
Agentic coding tools have delivered real value at Databricks, improving every velocity metric the company tracks and, on some teams, driving order-of-magnitude gains in output. But Databricks says nearly every company deploying AI coding tools at scale hits the same wall: exponentially growing costs that, left unchecked, threaten to overtake revenue and erase the efficiency gains the tools were supposed to deliver. In a blog post credited to Patrick Wendell, Akshat Bhatia, Vinay Gaba, Erich Elsen and Ivan Zhou, Databricks lays out techniques it says solve this problem, drawing on its own practice and on conversations with engineering teams at Stripe, Coinbase, Uber and Ramp. The goal, in the authors' framing, is a dual mandate: broad, low-friction access to AI tools for employees, held inside a roughly fixed cost envelope per user.
The single biggest lever, the post argues, is chasing what it calls the efficiency frontier rather than the intelligence frontier. Frontier labs mostly compete on the intelligence frontier, the highest-intelligence model, capable of novel math or cybersecurity work. Most day-to-day coding does not need that. What matters at scale is the efficiency frontier: the set of models with the best price for a given quality bar, which the authors say is advancing far faster than the intelligence frontier, with new models shipping almost weekly that beat prior ones on intelligence per unit price. Since public benchmarks poorly predict real-world coding performance, many companies build their own evaluations; Databricks published one such internal benchmark that found GLM models highly competitive on price and performance, and rolled GLM out to its developers as a result. New models do not always help: Stripe tested Opus 4.7 and found it did not meaningfully improve quality over Opus 4.6 while raising cost, and declined to make it available internally; Databricks says it saw a similar cost regression comparing Opus 5.0 to Opus 4.8.
Capturing efficiency-frontier gains requires tooling that can move between models, the post says, because harnesses such as Claude Code, Codex and Cursor are increasingly co-designed around specific model families, which risks the harness itself becoming a lock-in. Companies take one of two approaches: ask developers to manually switch harnesses when leadership wants to move spend to a cheaper model, which is workable but costly for an individual developer and becomes de facto lock-in if switching costs get too high; or adopt a meta-harness that gives developers one consistent interface while routing requests to different underlying harnesses and models behind the scenes. Databricks built its own meta-harness, Omnigent, and made it the default for its developers; some other companies interviewed have built custom internal meta-harnesses of their own. The authors also point to a growing body of research on automatic model and tool routing, which they say falls into roughly three categories, as a further way to squeeze out efficiency without asking users to pick a model themselves.
Despite the cost pressure, hard budgets, cutting a user off entirely once they hit a spend ceiling, are used only as a last resort at every company the authors spoke with. Cutting off access mid-task would hurt productivity that neither the company nor the employee wants to lose, and some of the highest spenders are simply the developers getting the most value from AI, so throttling them is self-defeating. Instead, companies favor visibility, near-instant, per-tool spend feedback, often paired with tips on cheaper models, plus progressively more friction as spend rises. The post includes a screenshot of a Databricks developer dashboard showing live spend as an example.
A typical coding request such as 'please investigate and fix this bug' triggers a chain of context-gathering: tool calls, codebase search, and skills or system information the company has wired in, so the user's original prompt ends up a negligible share of what actually reaches the model, and cost is dominated by that added context rather than by what the user typed. Techniques for cutting this context bloat are still new. Prompt caching matters once contexts get large: cache writes cost money but cached reads sharply cut per-inference cost, and the right cache settings depend on a company's specific workload. Databricks says tuning its harness and caching settings cut its generated tokens, and the associated cost, by almost 50%, with no observed quality degradation for developers, and that it expects further gains are possible.
Managing all of this, a shared model menu, unified spend observability across tools, and enforcement of context compression, requires a layer of infrastructure the authors call an AI Gateway. Databricks runs its own Unity AI Gateway for these functions and has released both Unity AI Gateway and Omnigent as open-source or free software; the post says thousands of companies now use these components every day. The authors summarize the playbook as: chase the efficiency frontier rather than the intelligence frontier, keep tooling model-flexible, route work to the cheapest capable model, replace hard budgets with visibility and progressive friction, and cut token overhead, arguing that exponential AI coding costs are a solvable engineering and governance problem rather than an inevitability. The post credits infrastructure leaders at Uber, Stripe, Coinbase and Ramp for commentary and review, and Thrive Capital for feedback on an early draft.
Key facts
- Databricks says tuning its harness and prompt-caching settings cut generated tokens, and associated costs, by almost 50%, with no observed quality loss.
- Stripe declined to deploy Opus 4.7 internally after finding it did not meaningfully beat Opus 4.6 on quality while costing more; Databricks reports a similar cost regression comparing Opus 5.0 to Opus 4.8.
- An internal Databricks benchmark found GLM models highly competitive on price and performance, leading Databricks to roll GLM out to its developers.
- Hard spending caps that cut off access at a threshold are used only as a last resort at every company surveyed; companies favor real-time spend visibility and progressive friction instead.
- Databricks has released its Unity AI Gateway and Omnigent meta-harness as open-source or free software, now used by thousands of companies daily.
Why it matters
AI coding tools are delivering real productivity gains, but their cost curve grows exponentially and, left unmanaged, can overtake the revenue or efficiency gains they were meant to produce. The post frames this as a solvable engineering and governance problem rather than a trade-off between access and cost, built around a dual mandate: keep AI tools broadly and easily available while holding aggregate spend to a roughly fixed amount per user. Its central idea, chasing the efficiency frontier (best price for a given quality bar) instead of the intelligence frontier (highest raw capability), reframes what 'better model' should mean for a company doing routine engineering work rather than research.
Who it affects
Engineering and platform teams running AI coding tools at scale: Databricks itself, plus Stripe, Coinbase, Uber and Ramp, who contributed commentary and review. It also concerns developers using harnesses such as Claude Code, Codex and Cursor, whose day-to-day tool choice and switching costs shape how easily their employer can move spend to cheaper models.
How to use it
The post's recommendations: build internal evaluations rather than trusting public benchmarks to pick models, since new releases do not always beat the incumbent (Stripe's Opus 4.7 test and Databricks' Opus 5.0 comparison both came back negative); adopt a meta-harness, either Databricks' open-sourced Omnigent or a custom internal one, so switching models does not mean asking every developer to switch tools; hand-tune prompt-caching settings for the company's workload; give users real-time spend visibility with tips on cheaper alternatives instead of hard budgets; and centralize model selection, cost tracking and context-compression enforcement in an AI Gateway, such as Databricks' own Unity AI Gateway, which it has also released as open-source or free software.
How solid is it
This is a company blog post from Databricks, not an independent or peer-reviewed study. It is credited to five named Databricks authors and says it draws on the company's own experience plus conversations with engineering teams at Stripe, Coinbase, Uber and Ramp, who are thanked for commentary and review, alongside Thrive Capital for feedback on an early draft. The post itself describes its numbers as directional, based on an informal survey of development teams, rather than rigorously measured; the concrete figures it does give, the almost-50% token reduction and the Opus comparisons, are Databricks' own internal findings, not independently verified.
Risks and caveats
The post promotes Databricks' own products, Unity AI Gateway and Omnigent, as the reference implementation of the practices it recommends, so it doubles as vendor content. Beyond the roughly 50% token-reduction figure, it does not disclose specific dollar savings or percentages for the other techniques it describes, nor the size of Databricks' overall AI coding spend, so readers cannot verify how large the aggregate savings actually are. It also does not name which models Databricks now uses in place of Opus 5.0 and 4.8, or give dates for when GLM was adopted or when the Opus comparisons were run.
“This efficiency frontier is advancing far faster than the intelligence frontier, with new models being released almost weekly that present better intelligence-per-unit-price than prior models.”
— Databricks, "Managing AI Coding Costs at Scale"