PrismML runs Bonsai LLM on Qualcomm smart glasses

At Qualcomm's Snapdragon Summit, the chipmaker showcased a new version of PrismML's 1-bit Bonsai LLM designed to run locally on smart glasses. The demonstration highlighted how the model operates on devices built on the Snapdragon AR1 Gen 1 Platform. This integration marks a significant step toward bringing sophisticated, real-time artificial intelligence directly to wearable hardware.

The smart glasses version of Bonsai is a 2-billion-parameter model specifically tuned for vision and language tasks. Developed by Caltech researchers and advised by UC Berkeley's Ion Stoica, PrismML specializes in shrinking larger models. The startup's compression technology reduces model size by 4x while retaining almost all of the original performance on standard benchmarks. This allows the 1-bit Bonsai LLM to process visual inputs and answer questions about what the wearer is looking at in real time without relying on cloud servers.

For AI developers and hardware practitioners, this development offers a viable path toward open-weight AI that runs entirely on-device. By maximizing the efficiency of existing edge computing power, developers can build applications that do not depend on the privacy promises or subscription costs of proprietary, cloud-based AI labs. This local execution model mitigates latency issues and addresses growing data privacy concerns associated with sending continuous video feeds to external servers.

While the technology demonstration proves the feasibility of running highly compressed, multi-billion-parameter models on lightweight wearable chips, consumers will have to wait to try it. No commercial smart glasses running PrismML's software have been officially announced yet. However, the partnership establishes a benchmark for what is possible on Qualcomm's dedicated augmented reality silicon.