Google Launches HEIR Compiler for Encrypted AI Models

Google has released Homomorphic Encryption Intermediate Representation, or HEIR, an open-source compiler and development toolchain. This technology is designed to translate standard, pre-trained machine learning models so they can perform inference on encrypted data. By utilizing homomorphic encryption, the toolchain allows servers to process ciphertexts and return encrypted results, ensuring that sensitive user data and proprietary AI models remain protected throughout the computation process.

For developers, HEIR acts as an abstraction layer to scale complex models across different cryptographic dialects. Currently, the workflow is not fully automated. A practitioner can write a program in Python, annotate the data types that require encryption, and use a tool called torch_mlir to export a PyTorch model into MLIR before compiling it for homomorphic execution. Google has already demonstrated the toolchain's utility in several areas, including hotword detection in audio streams, network intrusion identification, credit card fraud detection, and private content recommendation systems.

Despite the privacy benefits, practitioners must navigate significant performance trade-offs. Community discussions highlight that homomorphic encryption typically introduces a high computational overhead of about 10^3, or 1000x, for inference tasks. Specific benchmarks indicate that a 64 bit equality operation takes 80ms, addition and subtraction require 100ms, and division can take up to 8 seconds. However, some tasks remain highly viable; for instance, an image classification task might slow down from 1ms to 1s, which is acceptable for many real-world applications.

The performance outlook for large language models may be more encouraging because their workloads rely heavily on addition and multiplication. These operations are well-suited for homomorphic encryption, unlike branching operations which FHE handles poorly. Although Google has included benchmarking code in the HEIR repository, the company has not yet released official performance figures regarding its speed when applied to large language models.