Glonce

Google's ToolGrad flips tool-use data generation to answer-first

researchThe Glonce editorsGoogle Research🔥76Glonce rating76Impact70Novelty75Relevance95Surprise55Players70Credibility90
Google's ToolGrad flips tool-use data generation to answer-first

AI agents already show they can automate real-world tasks such as running a Google Search, reading local files or executing generated Python scripts, but doing this reliably requires LLMs trained on datasets that pair tool-use chains with the user queries that would trigger them. Google Research's post says the authors' own prior project, InstructPipe, relied on manual annotation for its evaluation data, which is impractical to scale for advanced LLM fine-tuning. Automated alternatives such as ToolBench and ToolACE work the other way around: they generate a hypothetical user instruction from a sampled pool of APIs, then send a depth-first search (DFS) agent to find, through trial and error, a tool-use solution that answers it. The authors call this query-first approach inherently inefficient, since its core idea is to distill valuable trajectories out of a complex agent exploration.

ToolGrad, introduced in a paper presented at ACL 2026, reverses that order. It first builds a ground-truth tool-use chain, then annotates the user prompt that would produce it. The authors argue an explicit tool-use solution carries more unambiguous information than a prompt does, so writing the matching query takes only one LLM step, instead of the baseline's full agent search. The method borrows a concept the authors call textual gradients from TextGrad, a separate technique that optimizes prompts by having an LLM critic write plain-text feedback and using that feedback to draft a better prompt. ToolGrad repurposes the same feedback loop to build API workflows instead of prompts, through four modules that run in sequence: propose, execute, select and update. Repeating this cycle produces one training sample: a user query, a verified API workflow and the model's final response.

To test it, the authors used ToolBench's database of more than 16,000 real-world APIs and compared ToolGrad against the original query-first, DFS-based approach on that same database. They report that ToolGrad produces more complex tool-use data, with a higher pass rate, at lower generation cost than that baseline. They then built a dataset called ToolGrad-500 from ToolBench and used it to fine-tune three sizes of the Gemma-3 model (1B, 4B and 12B parameters), naming the results ToolGrad-1B, ToolGrad-4B and ToolGrad-12B. These were evaluated on the Berkeley Function Calling Leaderboard (BFCL), a tool-use benchmark built on a different tool set than ToolBench, against unfine-tuned base models, the state-of-the-art proprietary models Gemini, GPT and Claude, and the state-of-the-art specialized tool-use models ToolACE and Hammer-2.1-7B. The authors report that the ToolGrad-tuned models outperform the baseline-trained ones and match the proprietary models on this out-of-distribution test. They add that the data-generation process itself achieves almost 100% pass rate, that relatively compact models can perform exceptionally well, and that student models can even surpass their teachers, without saying which specific models serve as the teachers. The post does not publish scores for any of these comparisons.

The post is bylined by Zhongyi Zhou, described there as a Research Scientist, and Ruofei Du, Interactive Perception and Graphics Lead at Google XR. A closing note credits Zhou with having primarily carried out the research during his tenure as a Visiting Researcher at Google, and names Kohei Uehara, Haoyu Zhang, Jingtao Zhou, Lin Gu, Zheng Xu and Tatsuya Harada as contributors, with Adarsh Kowdle and Shahram Izadi credited for strategic guidance and review. The authors say future work will scale the framework to larger and more dynamic API ecosystems, and explore extending it to continuous, on-the-fly learning for personalization.

Key facts

  • ToolGrad flips the usual recipe for tool-use training data: it builds a verified tool-use chain first, then writes the matching user query, a change the authors say needs one LLM step instead of the query-first baseline's full depth-first-search agent.
  • Tested on ToolBench's database of 16,000+ real-world APIs, ToolGrad's four-step process (propose, execute, select, update) is reported to produce more complex tool-use data at a higher pass rate and lower cost than the original query-first, DFS-based approach, though no baseline figures are published.
  • The authors fine-tuned three sizes of Gemma-3 (1B, 4B and 12B) on a ToolGrad-generated dataset called ToolGrad-500, then evaluated the results on the Berkeley Function Calling Leaderboard, a benchmark that uses a different tool set than training.
  • On that out-of-distribution benchmark, the authors report the fine-tuned models outperform baseline-trained models and match proprietary LLMs Gemini, GPT and Claude, and that data generation itself reaches almost 100% pass rate, all without publishing a single benchmark score.
  • ToolGrad, presented at ACL 2026, borrows the idea of textual gradients (plain-text LLM-critic feedback) from a separate prompt-optimization method called TextGrad, and was primarily carried out by Zhongyi Zhou, credited on the byline as a Research Scientist and separately as a Visiting Researcher at Google, with Ruofei Du of Google XR as co-author.

Why it matters

Feeding LLMs the tool-use training data they need has been a bottleneck: manual annotation, as in the authors' own InstructPipe project, does not scale, and the leading automated fix (ToolBench, ToolACE) still runs a two-step search, generating a hypothetical user instruction and then sending an agent to hunt for a matching tool-use solution by trial and error. ToolGrad targets that bottleneck by inverting the order: build the verified answer first, then write the one prompt that fits it, cutting the annotation step from a full agent search down to a single LLM call. The authors present this as a way to make training data for tool-using AI agents cheaper and higher quality to produce at scale, which matters as agentic workflows that call real tools become more common.

Who it affects

The clearest beneficiaries are teams building or fine-tuning models for tool use and agentic tasks, since ToolGrad is offered as a cheaper way to generate their training data. Google demonstrates this on its own Gemma-3 family, fine-tuning the 1B, 4B and 12B versions into ToolGrad-1B, ToolGrad-4B and ToolGrad-12B. The comparison also puts two existing specialized tool-use models, ToolACE and Hammer-2.1-7B, and the proprietary Gemini, GPT and Claude lines in the frame, since the fine-tuned Gemma-3 models are measured against all of them on BFCL. Whether any of this reaches outside developers yet is unclear: the post does not say whether ToolGrad's code, the ToolGrad-500 dataset or the fine-tuned checkpoints will be released.

How to use it

There is nothing to install from this post. ToolGrad is a data-generation method described in a paper presented at ACL 2026, built on top of ToolBench's library of more than 16,000 real-world APIs and, for the fine-tuning experiments, Google's Gemma-3 models. The post does not state whether the ToolGrad code, the ToolGrad-500 dataset it produced, or the fine-tuned Gemma-3 checkpoints are being made available outside the team, so for now the post says nothing more beyond the method the paper describes.

How solid is it

The work was presented at ACL 2026, a major peer-reviewed NLP conference, and the post names a specific author roster: Zhongyi Zhou, credited on the byline as a Research Scientist and separately as having done the work during a Visiting Researcher tenure at Google, and Ruofei Du of Google XR, with six named contributors and two credited for guidance and review. The evaluation design is reasonably broad, comparing fine-tuned models against base models, three proprietary LLMs and two specialized tool-use models on a benchmark, BFCL, that deliberately uses a different tool set than the training data, to test generalization. But the post itself is short on numbers: it states no pass rate, cost or benchmark figures for the baseline it compares against, and it gives no BFCL scores for any of the comparison groups, so the claims that ToolGrad-tuned models outperform baselines, match proprietary systems and reach almost 100% pass rate rest on the authors' prose, not on a published table.

Risks and caveats

Several specifics needed to judge the result are simply absent from the post: what the 500 in ToolGrad-500 counts (samples, tool-use chains or something else), which specific model or models are the teachers that student models are said to surpass, and which versions of Gemini, GPT and Claude were used as the proprietary baselines. The byline calls Zhongyi Zhou a Research Scientist while the closing credit says he carried out the work as a Visiting Researcher at Google, a discrepancy the post does not resolve. And because every comparative claim (pass rate, cost, benchmark performance) is qualitative rather than backed by a published figure, none of the reported gains can be checked against a number from this post alone.

“Our design achieves almost 100% pass rate in data generation, enables relatively compact models to perform exceptionally well, and shows that student LLMs can even surpass their teachers.”

— Zhongyi Zhou and Ruofei Du, Google Research blog