IBM Uses AI Agents to Run HuggingFace Models on Spyre

IBM has successfully deployed AI coding agents to bridge the gap between rapidly evolving open-source models and its new hardware. By generating a library of runtime patches called HF-adapters, the system has enabled stock HuggingFace Transformers to run on IBM's Spyre AI accelerator. Out of the 10,000 most-downloaded HuggingFace embedding models, just 13 distinct AI-written adapters managed to cover 7,960 of them. Ultimately, 6,804 of these models successfully passed their end-to-end execution tests on the Spyre hardware.

The Spyre accelerator, built on IBM's Artificial Intelligence Unit, uses a dataflow-driven architecture designed for low-power, high-throughput inference. Its memory and compute systems operate on fixed-size chunks called sticks, which are 128 bytes or 64 values in fp16 precision. Because standard PyTorch models do not naturally align with these hardware boundaries, the torch-spyre compiler backend requires translation. The AI-written adapters resolve these discrepancies at runtime without altering the underlying mathematical outputs of the models.

The automated patches range from simple operator swaps to complex structural changes. For instance, the gelu_new activation function used in decoders like GPT-2 and GPT-Neo contains a cube operation that fails to compile cleanly on Spyre when written as torch.pow(x, 3.0). The AI agent resolved this by substituting a mathematically identical multiplication of x * x * x. In other cases, the agents automatically padded vocabulary dimensions to ensure that large matrix multiplications could be divided evenly across the accelerator's processing cores, preventing compilation failures.

While AI agents excel at scanning massive codebases and drafting initial patches, human engineers remain crucial to the debugging process. Human supervision is required to localize silent failures and distinguish expected numerical drift from genuine compilation errors. This collaborative workflow allows the temporary adapter scaffolding to act as a diagnostic tool, exposing low-level bugs in the compiler stack while allowing developers to deploy models on day one.