Nvidia introduces AI router and open model to reduce task costs
Nvidia has released the NeMo Switchyard library and Nemotron 3.5 Lightning open model, aiming to cut AI agent task costs by up to two-thirds.
Nvidia is introducing NeMo Switchyard, an open-source library designed to intelligently route AI tasks to the most suitable models, alongside Nemotron 3.5 Lightning, a new open model optimized for high-volume, specialized agent tasks. The combined offering aims to significantly reduce operational costs for enterprises running AI agents.
According to Nvidia's internal tests, using Switchyard with models like Lightning can reduce benchmark task costs to approximately one-third compared to relying solely on advanced frontier models. Nemotron 3.5 Lightning itself offers up to four times faster output than comparable models in its class for specialized agentic tasks. This launch occurs amid a surge in competitive open-weight AI models from various global players.
The NeMo Switchyard library positions Nvidia against other open-source routing frameworks such as Not Diamond and RouteLLM. Nvidia's approach emphasizes a system-level solution, providing both the routing logic and optimized models under a unified open license, which the company believes offers a competitive advantage over model-only or router-only solutions.
"The power of a system of models, matching the right model to each step of the workflow, is key," stated Kari Briski, vice president of generative AI at Nvidia. Switchyard supports dynamic routing strategies that adapt to the evolving state of an agent's task, even factoring in cost considerations such as a model's typical output verbosity.
Early testing partners have reported substantial cost savings and performance improvements. LangChain reported a 74% cost reduction, while Ramp noted a 58% cost cut. Nvidia has integrated Switchyard with popular development frameworks and gateways, including LangChain and Kong AI Gateway, to facilitate adoption.