Essay argues you should run your own LLM proxy to see token usage
A blog post on jrdsgl.com, written in a blunt first-person voice, makes one argument: if you use LLMs, run your own LLM proxy on infrastructure you control. The author says the post's title captures the idea fully, though the source text does not give the title itself.
He starts from how most people use these models: through a frontend (a terminal UI, a GUI, a voice call) tied to a paid subscription from the provider of their choice. Your data goes in, their data comes back, and he says he is happy to pay for that. The provider then shows an analytics panel as proof of work. The author calls that a distraction. In his words, it is not token usage statistics with time series data meant for engineers, and he cannot make decisions from it. He expected the big labs to offer valuable insight to a curious user who digs into the Analytics panel, and says they do not.
He anticipates the objection that providers offer OTel export options. His answer is brief: that is one reason, and he can give five more off the top of his head. The list is:
- multiple providers and multiple models behind a single endpoint you control;
- logs of which key called what model, with what client, at what time, and with what status;
- virtual API keys, so you stop spreading your single main key across your work, agents and harnesses;
- infrastructure you control;
- the ability to switch model and provider extremely fast without setting everything up again (which, he adds, you will probably be doing for the third time).
He does not recommend a particular proxy. He only says to run it yourself somewhere you control, and to remember that you can modify it for your real needs. He has modified his own for better insight, custom MCP server endpoints and additional features, and says whatever you choose should give you data along the lines of what he shows.
His one concrete example is a cache miss during a coding session. He says that with his proxy he can pinpoint why it happened in 20 seconds by hand. He has not automated this, but the proxy he uses has an API, so automation is possible.
He closes with a call to own your infrastructure, your application visibility, your data pathing and your storage. He runs a proxy at home because it is fun and it is how he likes to learn, and at work because he feels he is not doing his job right otherwise. He adds that he may never understand how a large language model works well enough to whiteboard it in an interview, but he can wrestle it down and make sure it is instrumented while serving inference.
Key facts
- The author argues that anyone using LLMs should run their own LLM proxy on infrastructure they control, rather than rely on a provider's frontend.
- He says provider Analytics panels are not token usage statistics with time series data meant for engineers, and he cannot make decisions from them.
- He dismisses OTel export as one reason and lists five more: a single endpoint for many providers and models, logs of key, model, client, time and status, virtual API keys, infrastructure you control, and fast switching of model and provider.
- He says he runs a proxy at home and at work, has modified it for better insight, custom MCP server endpoints and extra features, and can pinpoint a cache miss in a coding session in 20 seconds by hand.
- He recommends no specific proxy, only that you run one yourself somewhere you control.
Why it matters
The post puts a practical gap into plain words. People pay for LLM access through a provider's app, and the usage display they get back is, in the author's view, a show of activity rather than data an engineer can act on. His proposed fix is to put a proxy you run between your tools and the providers, so that the logs, keys and routing sit under your control. It is an opinion piece, not a study, but it describes a setup that anyone working with several models or agents will recognise.
Who it affects
Mainly people who build with LLMs every day: developers using coding assistants, those running agents and harnesses, and engineers at work who are responsible for how models are served and instrumented. The author's examples range from personal projects to a large SaaS app, and he says he runs his proxy both at home and at work.
How to use it
The advice is to run an LLM proxy yourself, somewhere you control, and to pick one you can modify for your actual needs. Per the author, a good setup puts several providers and models behind one endpoint, logs which key called which model with which client at what time and with what status, and issues virtual API keys so your single main key is not scattered across work, agents and harnesses. It also lets you change model and provider quickly without rebuilding the setup. He says his own proxy exposes an API, which would make automated diagnosis of problems such as cache misses possible, though he has not built that. No specific product is named or recommended.
How solid is it
This is one person's opinion, written in the first person by an author who is not named in the source text. The one measurable claim, diagnosing a cache miss in 20 seconds by hand, is self-reported and comes without supporting figures. The post gives no costs, savings or benchmarks, and it does not explain why OTel export from providers falls short beyond the author's dismissal of it.
Risks and caveats
The argument rests on the author's own experience and preferences, and the post does not weigh the effort of running and maintaining a proxy against its benefits. Because no product is named, readers get a principle rather than a tested recipe. The author himself notes he has not automated cache-miss diagnosis, so part of the workflow he praises is still manual.
“Own your infrastructure, your application visibility, data pathing, and storage.”
— The post's author, jrdsgl.com