Hermes Agent Trains Physical Robot from Single Prompt

During Jensen Huang's keynote at GTC Berlin 2026, Nous Research demonstrated Hermes Agent running locally on an NVIDIA RTX Spark laptop. Prompted just once, the MIT-licensed agent harness self-installed and orchestrated a multi-stage robotics pipeline. It directed Nemotron to plan an obstacle course and enlisted GPT-6 Astra alongside FreeCAD, Blender, Omniverse, Newton, and Isaac Lab to construct the virtual terrain. Thousands of simulated Microduck robots trained in parallel on cloud GPUs before the verified policy was transferred directly to physical hardware onstage, where four units performed live commands.

The workflow relies on local inference powered by Nemotron 3.5 Bolt 74B, a mixture-of-experts design activating roughly 9 billion parameters per token. Using 2-bit quantization with quantization-aware training, the 74B model fits into 24GB of memory. It supports local runtimes like llama.cpp, vLLM, Ollama, Unsloth, and LM Studio, alongside NVIDIA's Switchyard router. To bridge the sim-to-real gap, the simulation ran at 1.25 million steps per second, generating an ONNX policy that ingests a 61-value observation vector to output 14 target joint positions over SSH.

This architecture changes how practitioners manage complex engineering workflows by dividing local orchestration from cloud-based compute. Hermes maintains persistent cross-session memory, integrates with platforms like Slack or Discord, and automatically writes executable SKILL.md files to store newly learned capabilities. Because these generated skills act as executable code, enterprise teams and robotics researchers must enforce strict sandboxing, approval tiers, and joint limits to prevent compromised agents from damaging physical hardware.