Google WikiSkill Helps AI Agents Learn From Mistakes

Google Research has developed WikiSkill, a framework designed to give AI agents a persistent database of their successes and failures. Instead of discarding operational data after a run, the system organizes an agent's workspace into three distinct tiers. The Raw Layer stores immutable execution traces and tool results. Above it, the Wiki Layer distills these traces into structured insights about failure patterns and successful strategies, growing continuously. At the top, the Skill Layer holds active procedural instructions that guide the agent. A Wiki Maintainer updates the wiki, a Skill Proposer suggests updates, and a gating mechanism validates changes, rolling back underperforming skills while keeping the wiki intact.

The researchers evaluated WikiSkill across five benchmarks covering math reasoning, web search, spreadsheet manipulation, document question-answering, and interactive tasks. Testing involved several models, including Gemma-4-31B, Gemini-3.5-Flash, and Qwen models of 4B, 9B, and 27B sizes, specifically Qwen-3.5-4B and Qwen-3.6-27B. The framework consistently outperformed previous skill evolution methods. On average, WikiSkill boosted Gemini-3.5-Flash performance from 49.5 percent to 68.1 percent, and Qwen-3.6-27B from 39.4 percent to 63.3 percent. On individual benchmarks, Gemini-3.5-Flash climbed from 33.0 percent to 72.6 percent on LiveMath and from 50.5 percent to 76.6 percent on SpreadSheet.

For practitioners, WikiSkill offers a practical workaround to the challenge of continuous learning without expensive model retraining. The framework allows smaller models running WikiSkill to match the performance of larger models that lack it. However, gains vary by task. While math and spreadsheet tasks saw significant improvements, long-context tasks like OfficeQA showed smaller gains, as smaller models like Qwen-3.5-4B struggled to execute complex, multi-step search strategies. Additionally, the researchers found that skills developed by one model can often transfer to another, though this must be evaluated on a case-by-case basis.