Goodfire Finds Qwen3-32B Neurons Are 90 Percent Clean

Goodfire researcher Nick Cammarata published new findings challenging long-held assumptions about neural network opacity, asserting that large language models are structurally far cleaner than researchers previously assumed. In preliminary audits of Qwen3-32B, Goodfire estimated that roughly 90% of neuron poles carry out one central task alongside small offshoots, approximately 7% handle two related functions, and only 2% to 4% remain unexplained. Critics note this benchmark does not represent a direct apples-to-apples jump from historical 1.7% interpretability figures, but the structural legibility holds across both dense and mixture-of-experts models. Testing extended up to trillion-parameter systems like Kimi K3, where researchers mapped neuron-only attribution graphs for complex decisions.

The research re-evaluates polysemanticity—the concept that networks pack unrelated ideas into single neurons via superposition. Cammarata argues that what previously appeared to be arbitrary concept-packing was actually recursive generalization, where a neuron branches into related meanings, such as moving from the letter sequence "iy" to "diy" and "do it yourself." The study identified functional motifs like summary neurons that collect context on punctuation; for instance, a comma neuron in a dense Qwen model was found to track Colorado-related data. Similar patterns appeared in older vision networks like InceptionV1, which holds over a hundred generalized pose neurons, as well as language models like Qwen3-8B. Activations varied in behavior, with specialized units dropping off sharply at 5% of maximum capacity.

For machine learning engineers and safety practitioners, these findings suggest that interpretability work may become significantly easier as models scale up. Rather than battling chaotic superposition, researchers can leverage the narrower, more predictable neuron functions found in larger networks to build clearer mechanistic maps. Although the discovery does not instantly solve AI alignment or provide an immediate comprehensive diagnostic tool, it offers practitioners a pragmatic blueprint to audit models directly by studying native neural circuits without first trying to untangle artificial representations.