Edge0 Runs 35B Qwen AI Model on iPhone with 2.9 GiB RAM

AI developer Edge0 has released an open-source inference framework capable of running a 35-billion-parameter language model on iPhone-class hardware. Published under the Apache 2.0 license on GitHub, the framework allows a 35B model derived from Qwen3.5-MoE to operate with a peak active-memory footprint of just 2.9 GiB. Edge0 has also released an 8B model checkpoint based on Ling 3.0, which requires only 1.0 GiB of active memory. Both checkpoints are currently available on Hugging Face.

To achieve this low memory footprint, the framework utilizes a streaming Mixture-of-Experts architecture. Instead of keeping the entire 23 GB 4-bit checkpoint in RAM, Edge0 memory-maps the expert files on local storage. It loads individual expert weights from the SSD on demand as each token moves through the model, caching frequently used experts and leaving the rest on disk. To combat the latency of reading from storage, Edge0 employs a small, trained prerouter head that predicts the experts needed for the next token one step ahead. This allows the system to overlap storage reads with active computation, boosting decode throughput by up to 59 percent.

For practitioners, this development shifts the economics of deploying large models on edge devices. The framework delivers a decode speed of 15 tokens per second while maintaining high accuracy. Thanks to quantization-recovery adapters, the compressed model averages only 3.9 points below the original fp16 base model on benchmarks. Currently, the framework only supports Apple's MLX backend, meaning it is restricted to iOS and macOS devices. However, the developers have placed CUDA support and enhanced agentic capabilities on their roadmap, which will eventually bring these memory-saving capabilities to a broader range of hardware.