Google and Meta launch native TorchTPU backend

Google and Meta have introduced TorchTPU, an open-source native backend enabling PyTorch to run directly on Google TPUs. Started in mid 2025 to move beyond 2020 era PyTorch/XLA and TorchAX tools, the project targets seamless hardware migration by changing standard device code directly to TPU targets. It features native support for ~450 Aten ops, custom kernels via JAX/Pallas or Helion DSL, and validation across 5,000 top Hugging Face models. Ecosystem compatibility covers vLLM, SGLang, TorchTitan, Ray, and TorchRec.

The initiative has verified major LLMs including Qwen, DeepSeek, Gemma, LLama, Mistral, GPT, OPT, BLOOM, Falcon, and StableML. Audio support encompasses Whisper, Wav2Vec2, HuBERT, and MMS, while vision coverage spans DinoV2, SigLIP, CLIP, Swin Transformer, ConvNeXt, ViT, ResNet, and DETR. Text encoders comprise BERT, ALBERT, Electra, RoBERTa, DeBERTa, T5, MT5, and Flan-T5. The architecture targets hardware such as TPU 8i, which packs 288 GB HBM at 8.6 TB/s with 19.2 TBps ICI, and TPU 8t, featuring 19.2 TBps ICI and 400 GBps DCN. TPU Matrix Multiply Units utilize 256x256 systolic arrays.

For practitioners, TorchTPU reduces compilation delays through flexible eager execution modalities: Strict, Debug, and DeferAndFuse, which speeds up operations by 50% or more. It manages hardware precision using native BF16 (1 sign bit + 7 mantissa bits + 8 exponent bits) and 19-bit TF32 (1 sign bit + 10 mantissa bits + 8 exponent bits), offering high and highest accuracy configurations at 3x and 6x MXU compute costs. Compilation overhead is further minimized across 3 cache tiers using a 192-bit key space composed of three 64-bit integers, giving engineers direct access to TPU hardware without restructuring existing PyTorch code bases.