Edge0 Streams 8B AI Model From SSD Using 1 GiB RAM

AI startup Edge0 has released Edge0-8B-A1B-preview, an 8-billion-parameter sparse mixture-of-experts (MoE) model designed to run on consumer hardware with minimal memory overhead. By streaming model weights directly from an SSD as they are needed for each token, the system requires only about 1 GiB of peak active memory. During testing on a Mac mini, the 4-bit quantized model achieved decoding speeds ranging from 23.9 to 25.3 tokens per second.

Built on inclusionAI's Ling 3.0 tiny base under the Apache 2.0 license, the model features 128 experts, K=8 routing, and a 128k context window. Although the total model contains roughly 7.9 billion parameters, the MoE architecture ensures that only about 1.2 billion parameters are activated per token. To minimize storage latency, Edge0 utilizes SSD expert offloading combined with a one-step-ahead route prediction mechanism, which boosts decode throughput by up to 59 percent.

For developers and practitioners, this release demonstrates the feasibility of running phone-class active memory deployments on local hardware, though the current implementation is limited to Apple Silicon via the MLX framework. The model ships with Recover-LoRA and prerouter adapters, allowing automatic loading via the Edge0 CLI. While the model is not yet optimized for agentic tool use or long-horizon tasks, it maintains high accuracy. Its average benchmark loss compared to the fp16 base is just 2.8 points, and it actually outperforms the base model on the MMLU-Pro benchmark.

Deploying the system requires accommodating the 4.55 GB checkpoint on disk, alongside the KV cache, runtime, and other process memory. Edge0 has also released a larger 35B sibling model alongside this preview, expanding the options for local, low-memory AI execution.