Google ToolGrad Framework Boosts AI Tool-Use Training
Google researchers have introduced ToolGrad, an 'answer-first' dataset generation framework that allows smaller open-source models to match proprietary LLMs in tool-use capabilities.
Google Research has developed ToolGrad, a synthetic data generation framework designed to teach large language models how to use external APIs. Unlike traditional query-first methods that generate a user prompt and then search for a matching tool-use path, ToolGrad reverses the workflow. It first constructs a verified, ground-truth tool-use chain and then generates the corresponding user query. This answer-first approach achieves a 99.8 percent pass rate, bypassing the high costs and low success rates of depth-first search agents.
The framework relies on textual gradients, a concept adapted from prompt optimization, to iteratively build API workflows from large libraries. ToolGrad coordinates four core modules: an API Proposer to suggest candidate tools, parallel API Executors to test them, an API Selector to choose the best step based on execution feedback, and an LLM Updater to adjust the synthetic query. To test the system, researchers used the ToolBench database of over 16,000 real-world APIs to generate a small dataset called ToolGrad-500 using the gemini-2.5-flash-lite model.
Using ToolGrad-500, the researchers fine-tuned Gemma-3 models at 1B, 4B, and 12B parameter sizes. Evaluated on the Berkeley Function Calling Leaderboard (BFCL), the resulting ToolGrad-12B model achieved a score of 83.1. This performance placed the open-source model on par with leading proprietary systems, nearly matching gemini-2.5-pro at 83.2 and outperforming claude-4.5 Opus at 82.8 and gpt-5 at 74.4. It also surpassed other specialized open-source models like ToolACE and Hammer-2.1-7B, and even beat its own teacher model, gemini-2.5-flash-lite.
For AI practitioners, ToolGrad addresses the scalability bottlenecks of manual annotation and inefficient search-based synthetic data generation. It demonstrates that developers can train highly capable, specialized tool-use agents using compact, cost-effective open-source models. By achieving high-quality results with a dataset of just 500 samples, the framework proves that targeted, high-fidelity synthetic data can dramatically lower the computational and financial barriers to deploying reliable agentic workflows.
This is our own summary of reporting by Google Research
