Glonce

SchemaRouter cuts RAG token use 9x without losing accuracy

researchThe Glonce editorsarXiv🔥72Glonce rating72Impact70Novelty75Relevance95Surprise45Players25Credibility85

Heterogeneous agentic retrieval-augmented generation (RAG) systems now orchestrate external APIs, internal databases, vector stores and graph stores at once. The authors argue this setup produces two costly failures: exposing every tool description to the LLM agent, or picking tools by vector similarity alone, either over-fetches (bloating payload size, token use and latency) or under-fetches (leaving out fields the query actually needs).

Their proposed fix, SchemaRouter, is a lightweight routing layer that represents tools, endpoints, parameters, response fields, domain concepts, units, provenance and license policies as a single schema graph. For each incoming query, a small LLM extracts the intent, the concepts involved and any source constraints. Field selection itself is then handled deterministically over the graph, through intent-group projection and concept-field matching with an alias layer, and the system emits an executable tool plan stating exactly which tools to call and which fields to retrieve.

The authors tested SchemaRouter on a materials-science benchmark of 110 queries against two baselines: fetch-everything, which retrieves full payloads, and prompt-all, which exposes all tool descriptions to the agent. SchemaRouter reached answer accuracy of 0.71, matching fetch-everything within overlapping confidence intervals and exceeding prompt-all's 0.66, though those intervals overlap too. It used 227 retrieved-context tokens per query versus 2,066 for fetch-everything, and delivered 2.7x lower end-to-end latency than prompt-all. It also posted the best tool-exact rate (0.93) and parameter validity (1.0) among the compared methods, and grounded provenance and license information in 62 percent of its answers, versus roughly 0 percent for every baseline.

A follow-up test found that simply minimizing the number of selected fields backfires: it drops answer accuracy to 0.56 while saving negligible tokens. Using recall-preserving projection instead, as SchemaRouter does, restores top accuracy. The authors present the result as evidence that schema-graph routing gives heterogeneous agentic RAG efficiency, schema-size-independent scaling and verifiable, license-grounded answers without trading away accuracy.

Key facts

  • SchemaRouter uses 227 retrieved-context tokens per query versus 2,066 for the fetch-everything baseline, a roughly 9x cut.
  • It delivers 2.7x lower end-to-end latency than the prompt-all baseline.
  • On a 110-query materials-science benchmark it scores 0.71 answer accuracy, matching fetch-everything and exceeding prompt-all's 0.66, though the confidence intervals overlap in both comparisons.
  • It posts the best tool-exact rate (0.93) and parameter validity (1.0) among the methods compared.
  • It grounds provenance and license information in 62 percent of answers versus roughly 0 percent for all baselines; minimizing selected-field count instead of using recall-preserving projection drags accuracy down to 0.56.

Why it matters

Agentic RAG systems increasingly have to juggle APIs, internal databases, vector stores and graph stores in a single pipeline. The authors' framing is that today's two common approaches both fail: dumping every tool description into the prompt, or picking tools by vector similarity, either over-fetches (more payload, tokens and latency) or under-fetches (missing fields the answer needs). SchemaRouter's pitch is that treating the tool landscape as a schema graph and computing field selection deterministically avoids both failure modes at once, cutting cost while adding something the baselines do not provide at all: grounded provenance and license information.

Who it affects

The design targets teams building heterogeneous agentic RAG systems that pull from multiple, differently shaped sources, external APIs, internal databases, vector stores and graph stores, in the same query pipeline. The evaluation domain is materials science, where provenance and license grounding matter for trusting an answer, but the problem the authors describe (over-fetching versus under-fetching across mixed tool types) is not specific to that field.

How to use it

SchemaRouter models tools, endpoints, parameters, response fields, domain concepts, units, provenance and license policies as one schema graph. A small LLM handles only intent extraction, concepts and source constraints; which fields actually get retrieved is decided deterministically through intent-group projection and concept-field matching with an alias layer. The output is an executable tool plan naming exactly which tools to call and which fields to pull, rather than a free-form prompt or a similarity-ranked list.

How solid is it

The results come from a single benchmark of 110 materials-science queries. SchemaRouter's 0.71 accuracy matches the fetch-everything baseline within overlapping confidence intervals and edges out prompt-all's 0.66, though that comparison's intervals overlap too, so the accuracy advantage over prompt-all is not statistically decisive on this data. Its token and latency wins are large and unambiguous: 227 versus 2,066 tokens against fetch-everything, and 2.7x lower latency than prompt-all. An ablation backs the design choice directly: minimizing selected fields to save tokens instead cuts accuracy to 0.56, while the recall-preserving projection SchemaRouter actually uses restores top accuracy.

Risks and caveats

The evaluation is limited to one domain and one 110-query benchmark, and the accuracy gains over prompt-all rest on confidence intervals that overlap, so the paper's own numbers do not establish statistical significance there. The text does not name which small LLM handles intent extraction, nor does it give author names, institutional affiliations, publication venue or a timeframe for real-world deployment, so how the system would generalize outside materials science, or when it might see production use, is not stated.