Cua Speeds Up llama.cpp Inference in macOS VMs
Virtualization startup Cua has released a compatibility shim that bypasses macOS VM graphics limitations, boosting llama.cpp local LLM inference speeds by up to 16 times.
Cua has released a process-scoped Metal capability shim to accelerate local AI workloads inside virtual machines. When running macOS guests via Apple's Virtualization.framework, the virtual GPU typically reports conservative capabilities, such as an Apple 5-era family and a 32 KB threadgroup memory limit. This forces llama.cpp to run slower GPU code. Cua's compatibility layer intercepts these queries inside a guest process, reporting support up to Apple family 9 (1009) and raising the reported maximum threadgroup memory to 64 KB. This allows llama.cpp to select modern Metal kernels, including SIMD-group reduction, SIMD-group matrix, and bfloat16 paths.
Testing on an Apple M1 Ultra host with a 48-core GPU running macOS 26.6.1 and a Tahoe guest VM (macOS 26.5.2, 8 vCPU, 16 GiB) in Lume 0.5.1 used the llama.cpp b10167 release. For TinyLlama 1.1B Chat Q4_K_M, prompt processing jumped 11.08 times from 432 to 4,787 tokens per second (98% of bare-metal speed), while token generation sped up 16.36 times (72.06% of bare-metal). With Google's 6.98 GB Gemma 4 12B QAT Q4_0 model, prompt processing increased 7.20 times from 71.66 to 515.76 tokens per second (99.59% of bare-metal), and token generation rose 14.54 times from 3.41 to 49.67 tokens per second (94.82% of bare-metal).
Testing Meta's 16.76 GB Muse Glimmer 30B Q4_K-M GGUF in a 64 GiB guest using llama.cpp b10359 showed a 7.55-fold increase in processing a 512-token prompt (from 25.83 to 194.97 tokens per second) and an 8.87-fold increase in generating 128 tokens (from 2.38 to 21.08 tokens per second). However, testing MLX-LM 0.31.3 with mlx-community/Llama-3.2-3B-Instruct-4bit on MLX 0.32.0 showed flat performance. During ablation, advertising MTLGPUFamilyMetal3 caused issues, so the shim limits changes to Apple-family enums.
For developers, this bridges the gap between virtual machines and bare-metal hardware. Users can enable this feature by configuring the ForceUnrestrictedDeviceFeatureLevel boolean to true in their macOS settings and injecting the LumeMetalCapabilities-arm64.dylib library. This allows developers to run high-performance AI development stacks on Apple Silicon without sacrificing the security of virtual machines.
This is our own summary of reporting by Hacker News
