CPU-ARC paper claims LLM reasoning can be cached and run on CPUs
The paper starts from a complaint about large language models: they "reinvent the wheel all the time". According to the authors, models conflate task-solving with executing the solutions, so every problem gets solved from scratch, even when the task has been solved before.
The authors say they show that once a task has been reasoned through, applying its solution to a new problem instance can be done very efficiently and with little involvement from language models.
To get there, they introduce agentic reasoning caching, a method for constructing robust CPU-executable agentic reasoning caches (ARCs) from LLM reasoning traces. In other words, the expensive reasoning is done once by a language model, and the result is stored as something that runs on an ordinary processor.
The abstract makes several claims about these caches. ARCs show on-par performance with frontier LLMs across a range of difficult mathematical and scientific tasks. They are fully explainable, continually trainable and robust to input perturbations. And they are 16-291x cheaper to invoke than the source model, the model whose reasoning traces they were built from.
The text available is the abstract alone. It gives the 16-291x figure only as a range and does not break it down by task or model.
Key facts
- The paper introduces agentic reasoning caching, a method for building CPU-executable agentic reasoning caches (ARCs) from LLM reasoning traces.
- The authors argue LLMs conflate task-solving with executing solutions, which forces them to solve every problem from scratch.
- They claim ARCs perform on par with frontier LLMs across a range of difficult mathematical and scientific tasks.
- They claim ARCs are 16-291x cheaper to invoke than the source model, and are fully explainable, continually trainable and robust to input perturbations.
Why it matters
The paper targets a waste the authors say is built into current LLM use: a model re-solves each problem from scratch because it mixes up working out a solution with carrying it out. If a solution found once can be applied to new instances with little involvement from a language model, repeated tasks could stop needing a full LLM call each time. That is the authors' claim, and the 16-291x lower invocation cost is the headline number behind it.
Who it affects
The idea concerns anyone running a frontier LLM repeatedly on the same kind of mathematical or scientific problem, where the invocation cost of the source model is the burden. The abstract does not name specific users, products or deployments.
How to use it
No code, model release or dataset release is mentioned. Nothing in the abstract describes a procedure a reader could follow today; the method for constructing ARCs from reasoning traces is not described there.
How solid is it
This is a preprint listed on Hugging Face Papers, and all claims come from its abstract, which is the only text available. 'On-par performance' is not quantified, and the 'frontier LLMs' and 'source model' are not identified. The 16-291x figure is given only as a range with no breakdown per task or model, and 'cheaper to invoke' is not defined as money, compute or time. No authors or institutions are named. The claims are the authors' own and have not been independently checked here.
Risks and caveats
Read the numbers as claims, not results. Without named benchmarks, baselines or accuracy figures, it is not possible to judge how broad the 'on-par' finding is. The tasks covered are mathematical and scientific; the abstract says nothing about other kinds of work. The 16-291x range is wide, and where a given task falls in it is not stated.
“Large language models reinvent the wheel all the time.”
— Opening line of the paper's abstract, Hugging Face Papers