Nvidia Secures DeepSeek-R1 Inference on Blackwell GPUs

Nvidia has introduced confidential computing optimizations to its TensorRT-LLM framework, enabling secure, hardware-encrypted AI inference on its Blackwell architecture with minimal performance loss. In benchmark tests conducted by Nvidia's performance engineering team, an eight-GPU Nvidia DGX B200 system running the nvidia/DeepSeek-R1-0528-NVFP4 model retained 96.1% to 98.2% of its standard throughput when confidential computing was enabled. The latency overhead, measured as Time Per Output Token, remained within a modest 1.2% to 4.3% of the unencrypted baseline.

The evaluation utilized a demanding workload configuration designed to expose security overhead, featuring a long input context of 32K tokens, an output of 1K tokens, and concurrency levels ranging from 1 to 16. The system ran on an Intel TDX platform with Ubuntu 25.10 as the host operating system and Ubuntu 24.04.4 LTS as the guest. Hardware parameters included a 1,000-watt GPU power limit, 256 guest vCPUs, and two NUMA nodes. The software stack consisted of CUDA 13.2, NCCL version 2.30, OpenSSL 3.6.0, and TensorRT-LLM version 1.3.0rc22 with an FP8 KV cache and tensor parallelism set to eight.

To achieve these results, Nvidia developers modified how TensorRT-LLM handles data movement and scheduling under confidential computing, where the GPU cannot directly access protected virtual machine memory. Instead of relying on pinned memory, the framework now uses pageable memory for host-to-device transfers. It also offloads repeated token and sampling-data readbacks to an asynchronous worker thread to prevent scheduler blocks. Furthermore, because standard CUDA events produce unstable timing signals in secure environments, the framework's autotuner was updated to measure performance using the GPU globaltimer.

Finally, because NVLink SHARP multicast is unavailable in B200 confidential computing configurations, the framework was adapted to detect this limitation and dynamically select communication algorithms that minimize latency. These full-stack engineering adjustments demonstrate that secure, private production AI does not require sacrificing high-performance capabilities.