NVIDIA Launches SkillEvaluator to Benchmark AI Agents

NVIDIA has introduced SkillEvaluator, an open-source tool designed to assess how verified skills affect AI agent performance. These skills are signed capability descriptors that instruct agents on how and when to use specific NVIDIA products. To ensure quality, SkillEvaluator runs a three-tier evaluation process: Tier 1 checks safety and structure, Tier 2 analyzes distinctiveness using embedding similarity, and Tier 3 runs live evaluations in isolated sandboxes using the Harbor framework.

In an August 12, 2026 benchmark snapshot of over 300 verified skills across more than 30 products, the tool demonstrated significant performance improvements. Evaluated on Claude Code and OpenAI Codex, the baseline scores without skills were 46 for Correctness, 42 for Discoverability, 39 for Effectiveness, 43 for Efficiency, and 97 for Security. With the skills installed, scores rose to 87 for Correctness (a 41-point lift), 82 for Discoverability (+40), 78 for Effectiveness (+39), 78 for Efficiency (+35), and 98 for Security (+1). This represents an average gain of 31 points overall, and 39 points when excluding Security.

The benchmark revealed that Claude Code achieved a higher average Skill Lift of 34 points (42 excluding Security) compared to OpenAI Codex's 29 points (36 excluding Security). Beyond these platforms, OpenClaw is piloting the tool on ClawHub, while Nous Research integrated it into Hermes Agent alongside a SkillSpector advisory scan that runs in 1.4 to 1.5 seconds.

For developers, the evaluation shows that the specific product domain and dataset design influence performance gains far more than the choice of agent harness, with per-product Skill Lift ranging from +2 to +46 points. Furthermore, the tool tracks resource efficiency. For example, the jetson-optimize-memory skill slashed token usage by 76.9 percent (from 617,306 to 142,540) and execution time by 53.7 percent. Conversely, the cuopt-install skill increased tokens by 120.3 percent and execution time by 20.8 percent, highlighting where developers need to focus their optimization efforts.