Glonce

ToolSpace builds MCP agent training environments from real tool-call traces

researchThe Glonce editorsHugging Face🔥58Glonce rating58Impact45Novelty60Relevance95Surprise30Players12Credibility70
ToolSpace builds MCP agent training environments from real tool-call traces

A paper listed on Hugging Face Papers proposes ToolSpace, a framework for generating environments in which language-model agents practise long-horizon tool use over MCP (Model Context Protocol) services. The text available is the abstract.

The authors start from a criticism of existing MCP environment-scaling methods. Those methods are data-centric: they first synthesize a database and a scenario, then derive tools and tasks from them. The authors say this has three consequences: tool interfaces may not match real ones, tool compositions are not grounded in execution, and tasks rarely span more than one environment.

ToolSpace reverses the order and starts from the tools. It mines data, state and retry dependencies from adjacent calls in execution traces of real MCP services and open agent corpora, using deterministic rules rather than LLM judgment. The resulting weighted tool-composition graph is clustered into environments. Databases are synthesized only for state-mutating environments; query and stateless tools replay recorded returns. ToolSpace then runs state-aware random walks with rollback and decompiles each chain into a natural-language task with assertion-based checks. It keeps only the samples that replay successfully from the initial state.

The results come in two parts. First, the pipeline was applied to AutomationBench and tau3-Bench Banking to produce harder "Pro" versions of both. On those versions, 7 frontier models drop by up to 38.2 points. Second, across seven long-horizon tool-use benchmarks, ToolSpace raises Qwen3.6-35B-A3B from 23.0 to 41.0 average score, or 43.2 with RL. GLM-5.2 goes from 44.6 to 56.6, which the abstract describes as on par with Claude Opus 5.

The authors also report that ablations show execution-witnessed edges clearly outperform LLM-judged ones, and that performance keeps improving with data and environment scale.

Key facts

  • ToolSpace is tool-centric: it builds MCP environments from tool-call traces of real services and open agent corpora, instead of first synthesizing a database and scenario.
  • Dependencies between calls are mined with deterministic rules rather than LLM judgment; kept samples must replay successfully from the initial state.
  • On the harder Pro versions of AutomationBench and tau3-Bench Banking, 7 frontier models drop by up to 38.2 points.
  • Across seven long-horizon tool-use benchmarks, Qwen3.6-35B-A3B rises from 23.0 to 41.0 average score (43.2 with RL).
  • GLM-5.2 rises from 44.6 to 56.6, described as on par with Claude Opus 5; ablations favour execution-witnessed edges over LLM-judged ones.

Why it matters

Agents that chain many tools across MCP services need training and test environments that look like the real thing. The authors argue that data-centric synthesis produces tool interfaces that may not match real ones and tasks that rarely cross environments. ToolSpace tries to fix this by grounding environments in recorded executions and by building multi-tool compositions from observed dependencies between adjacent calls.

Who it affects

Teams that train or evaluate tool-using agents on MCP services are the direct audience, along with builders of agent benchmarks. The abstract frames the work around long-horizon tool use and reports results on two named models, Qwen3.6-35B-A3B and GLM-5.2, plus 7 unnamed frontier models on the Pro benchmarks.

How to use it

The abstract describes the pipeline: mine dependencies from execution traces, cluster the tool graph into environments, synthesize databases only where state is mutated, then generate tasks by random walks with rollback and keep the ones that replay. It also describes the harder Pro versions of AutomationBench and tau3-Bench Banking. No code, dataset release, license or compute cost is mentioned.

How solid is it

This is a preprint listing with an abstract only, and the figures are the authors' own. The 38.2-point figure is a maximum drop ('up to'), and the abstract does not say which model or benchmark it comes from. The 7 frontier models and the seven benchmarks are not named, apart from AutomationBench and tau3-Bench Banking. The abstract does not say how the RL variant was trained. Claude Opus 5 appears only as a comparison point and its score is not given.

Risks and caveats

The abstract mentions no limitations or failure cases. The Pro benchmarks were built by the same pipeline whose training gains are reported, so readers will want details from the full paper. The claim that ToolSpace's harder benchmarks and the gains transfer across seven benchmarks rests on the authors' evaluation, which cannot be checked from the abstract alone.

“We propose ToolSpace, a tool-centric framework that reverses this order.”

— ToolSpace paper abstract, Hugging Face Papers