PyTorch 2.14 Launches with NVGEMM and Python 3.15 Support

The PyTorch Foundation has officially launched PyTorch 2.14, delivering 2,995 commits from 487 contributors. This release matures the framework's compiler technology, introducing NVGEMM to bring CuTeDSL-generated CUTLASS kernels to the Inductor compiler. NVGEMM enables epilogue fusion, scaled and NVFP4 GEMM, and grouped-reduction epilogues. Additionally, Inductor now enables compute and communication overlap by default, automatically boosting GPU utilization during distributed training. For control flow, the new torch.switch operator replaces nested conditionals with multi-way branching, while torch.while_loop can now be captured in CUDA graphs.

Distributed training receives a major upgrade with the new nccl2 backend, ported from torchcomms. This backend implements the full collective contract with nonblocking communicators and eager communicator splitting. Fault tolerance is now a first-class c10d concept, introducing in-place process-group reconfiguration for Gloo and nccl2 backends so clusters do not have to restart after a single node failure. The Flight Recorder collective trace buffer has also been decoupled from NCCL, allowing it to diagnose hangs on any backend. Furthermore, DTensor sharding rules have expanded to cover 1,239 operators, up from 585 in January 2026.

Apple Silicon users gain native linear algebra support, including Jacobi-kernel SVD, eigh, QR, and Cholesky. A five-part reduction rewrite and a migration of operators like index_add and conv3d from MPSGraph to native Metal kernels significantly reduce overhead. Notably, a routing fix for single-token decode shapes resolves an 8.5x slowdown on bf16 and fp16. Beyond Apple hardware, PyTorch 2.14 expands platform support to AMD ROCm 7.14 wheels, Intel XPU native graph capture, and NVIDIA's next-generation Rubin architecture.

This release introduces binary support for Python 3.15, including the free-threaded 3.15t build across CPU, CUDA, ROCm, and XPU. However, torch.compile is not yet supported on Python 3.15, meaning users requiring compilation should remain on Python 3.14 or earlier. Finally, TorchVision 0.29.0 is now ABI stable with respect to PyTorch 2.14, ensuring compatibility with future 2.x releases without requiring synchronized updates.