Nvidia Releases NeMo Switchyard to Slash AI Agent Costs

Nvidia has introduced NeMo Switchyard, a free and open-source model routing library aimed at optimizing the cost of running AI agents. In typical production environments, long-running agents often waste resources by sending simple tasks like tool calls, result validation, and subagent delegation to expensive frontier models. Switchyard addresses this inefficiency by dynamically directing each step of an agent's workflow to the most appropriate and cost-efficient model available.

In testing conducted by LangChain across 145 multi-turn tasks, the library achieved a 74 percent reduction in costs. It managed this by sending only 7 percent of total calls to a premium frontier model, accepting a minor accuracy tradeoff of roughly 6 points. Alongside the library, Nvidia launched Nemotron 3.5 Lightning, a 30-billion-parameter mixture-of-experts model with 3 billion active parameters, specifically designed to handle the high-volume execution layer of these agent pipelines.

The software includes three tuning-free routers that work out of the box: an LLM classifier, a stage router, and an escalation router. Additionally, a tunable prefill router leverages internal model data for learned routing decisions. Switchyard is compatible with APIs from OpenAI, Anthropic, and Responses, allowing developers to integrate it into existing agent setups with minimal code modifications. It already features active or upcoming integrations with platforms like LangChain, LiteLLM, Kong, Cognition's Devin, Ramp, Cadence, and Siemens.

For practitioners, this tool shifts model selection from a manual integration chore to an automated process. It capitalizes on the reality that different models excel at different tasks; for instance, while DeepSeek V4 leads overall on the Terminal-Bench Hard benchmark, Kimi K2.6 performs better on machine learning and reinforcement learning, and Qwen3.5 397B A17B dominates in math and science. However, because Switchyard is currently pre-alpha software, Nvidia warns that its APIs and algorithms will change significantly before its official 1.0 release, meaning it is not yet recommended for production environments without thorough evaluation.