Magic Matches DeepSeek V4 Pro Base Using 50x Less Compute

AI startup Magic has unveiled a pretraining methodology that matches the performance of DeepSeek V4 Pro Base while utilizing roughly 50 times fewer FLOPs. This training run cost approximately $500,000 on GB200 hardware, about half the pretraining compute required for GPT-3. When Magic scaled this recipe 10 times further to a budget of around $4 million, the resulting model outperformed all publicly available open base models on perplexity evaluations across code, math, and research text. Under the standard DeepSeek V4 Pro recipe, achieving this capability level would have cost upwards of $100 million.

To ensure rigorous benchmarking, Magic measured bits-per-byte loss on held-out data. The evaluation sets included private codebases, chain-of-thought math walkthroughs generated by Kimi K3, and low-citation research papers. The team applied aggressive decontamination, removing any document with a matching 96-character window or high Jaccard similarity. They validated logprob measurements across vLLM and SGLang on GB200 and GB300 systems, partnering with Fireworks to verify the baselines. Notably, the team observed that Nemotron 3 outperforms DeepSeek V4 Pro on base-model bits-per-byte across these domains.

The efficiency gains stemmed from tens of compounded changes across model architecture, optimizer, training objective, and data curation. To test post-training viability, Magic initiated a short reinforcement learning run with a 16,000 chain-of-thought token budget directly from the base model. On a private competition math evaluation harder than AIME, their V5 (e24) model achieved a 72 percent pass@1 rate, placing it ahead of DeepSeek V4 Pro at 65 percent and just behind Kimi K3 at roughly 75 percent.

Furthermore, the e24 model surpassed 90 percent pass@1 on AIME26, and reached 100 percent pass@16, using only 0.2 percent of its pretraining compute budget and averaging just over 6,000 chain-of-thought tokens. While competitors like Kimi K3 and Muse Spark demonstrate 2.5 times and 3.3 times efficiency gains over Kimi K2, Magic's progress represents an order of magnitude larger leap. The startup, which claims to be the smallest team training trillion-parameter models, plans to focus next on long-horizon reinforcement learning and an eventual public model release.