Meta Integrates Networking Into New MTIA 300 AI Chip

Meta has introduced the MTIA 300, its latest custom-designed silicon accelerator tailored specifically for training ranking and recommendation models. Unlike large language models that demand massive raw compute, recommendation models spend significant time transferring data between accelerators because embedding tables represent over 99% of their parameters. To solve this bottleneck, Meta integrated networking directly into the chip package, eliminating the need to route data across a standard PCIe bus.

The MTIA 300 features two network chiplets containing six custom 800 Gbps RDMA network interface cards (NICs) each, delivering a total input/output bandwidth of 1.2 TB/s. These 12 NICs manage both scale-up rack communication and scale-out traffic between racks. To prevent communication tasks from draining compute resources, the chip includes 16 dedicated message engines that operate independently of the main compute grid. This architecture allows massive matrix operations and collective communication to run simultaneously, resulting in less than 0.5% degradation in compute throughput, compared to a drop of more than 20% on traditional GPU architectures.

The hardware operates alongside HCCL, Meta's proprietary collective-communication library, which compiles operations into subgraphs that the message engines execute autonomously without host CPU intervention. In production environments, HCCL achieves up to 940 GB/s of communication bandwidth within a single rack. When training a 150-billion-parameter recommendation model across 40 accelerators, the MTIA 300 cut total communication time by 3.9x compared to an equivalent GPU cluster.

For AI practitioners, this custom architecture offers a highly optimized alternative to general-purpose GPUs for data-heavy recommendation workloads. Meta already deploys hundreds of thousands of MTIA chips for inference and plans to release four more generations over the next two years. The company is co-developing future iterations with Broadcom while continuing to source hardware from Nvidia and AMD as part of a multi-vendor strategy.