Laya-MLX Runs AI Decisions in 7ms on Apple Silicon

Developers can now run the Laya family of typed decision models locally on Apple hardware using the newly released laya-mlx package. This Apache-2.0 licensed independent port brings pre-converted FP16 weights to Hugging Face, allowing installation via a simple pip install laya-mlx command. By targeting Apple's native MLX framework, the package eliminates the need for a PyTorch runtime or external cloud APIs, keeping all inference entirely on-device.

Unlike traditional generative models that rely on token-by-token decoding and complex JSON parsing, Laya utilizes a bidirectional encoder. It features dedicated decision heads for choice, score, and P(true) propositions, returning calibrated probabilities directly. For practitioners, this architectural shift bypasses the latency and parsing errors associated with text generation, making it highly suitable for real-time applications.

Performance benchmarks on an M3 Max system equipped with a 40-core GPU and 128 GiB of unified memory demonstrate remarkable speed. The model achieved a median single-query latency of 13.4 milliseconds for short English decisions and just 7.4 milliseconds for multilingual tasks. When processing in batches of 50, the multilingual checkpoint reached a throughput of 395 questions per second. Crucially, peak memory consumption remained under 1 GiB for both configurations.

The port maintains high fidelity, matching the upstream PyTorch implementation on all 63 out of 63 validation questions. To showcase its real-time capabilities, the project includes a Snake game demo that runs locally at approximately 60 decisions per second. This demo displays per-move probabilities and incorporates a cycle-safety layer to reject unsafe proposals before they are executed.