Intern-S2-Mobius Speeds Up Inference Fourfold

Researchers have unveiled Mobius-v0, a novel foundation model architecture designed to separate global memory storage from iterative reasoning modules. This decoupling aims to address the inefficiencies of traditional Transformer architectures by isolating knowledge compression from the computational steps required for reasoning. The design utilizes a globally shared Feed-Forward Network as memory to store knowledge vectors, while multiple self-attention modules act as reasoners to perform compositional reasoning.

In this setup, the system uses hidden states as both a cache and a carrier. The reasoning modules repeatedly query the shared memory to retrieve the necessary knowledge vectors, which are then transmitted back to the reasoning operators. By separating these two core functions, the Mobius architecture achieves significantly better knowledge compression and reasoning efficiency than standard unified architectures.

The researchers demonstrated the viability of this approach with two distinct models. First, a 7B parameter Mobius-v0 model trained from scratch matched the downstream performance of a standard 7B Transformer baseline while requiring only 62.6 percent of the baseline's training data. Second, they developed Intern-S2-Mobius by continually pretraining the architecture from the Qwen3.5-35B model. This larger model achieved comparable downstream scores to its predecessor while delivering an impressive near-fourfold speedup in end-to-end inference.

For AI practitioners, this architectural shift offers a dual benefit of reduced training overhead and vastly accelerated deployment. The ability to achieve baseline-level accuracy with nearly 40 percent less training data drastically lowers the computational cost of training new models from scratch. Meanwhile, the fourfold increase in inference speed for continually pretrained models like Intern-S2-Mobius makes hosting large-scale models much more economically viable for real-time applications.