A neuron in a neural network is polysemantic when it activates for several unrelated concepts. The word comes from Greek, where poly means "many" and sēma, the root of semantic, means "sign". If a neuron only fired for one thing, like the colour red or Python code, it would be monosemantic, and you could read what the model is doing straight off it. Real networks aren't that tidy, and the classic example is a neuron in the vision model InceptionV1 that responds to cat faces, the fronts of cars and cat legs (Olah et al., 2020). Language models do it too, and in a small transformer Bricken et al. (2023) found a neuron that responds to a mix of academic citations, English dialogue, HTTP requests and Korean text.
Why it happens
The main explanation is superposition (though not everyone agrees, more on that below), where a model represents more features than it has neurons by storing each feature as a direction spread across many neurons, rather than giving it a neuron of its own (Elhage et al., 2022). Two things make that possible. First, there are far more concepts ("features") a language model could use than there are neurons in a layer, so it can't give each one its own neuron. Second, most features are sparse, meaning they're rarely active at the same time. Portsmouth Football Club and Pythagoras' theorem almost never turn up in the same sentence, so the model can "let" their directions overlap, and they'll rarely get in each other's way. I want to be careful with the word "let", because it suggests the model chooses how information is encoded in its weights. In reality the model doesn't choose anything, and the overlap is just where training ends up.
Compressing so many concepts into a fairly small number of neurons has a cost. When two features whose directions overlap are active at the same time, they interfere with each other, and the model has to filter out that noise. Elhage et al. found that their toy models only switch to superposition once the features are sparse enough, and that the features then settle into neat shapes, such as five features in two dimensions forming a pentagon. Each neuron picks up every feature whose direction points partly along it, which is what makes it polysemantic.
Five features, two neurons
What each neuron responds to
Why it's a problem
Polysemanticity is one of the main obstacles for mechanistic interpretability, the part of interpretability research that tries to work out, from a network's weights and activations, the algorithms it has learned. When a polysemantic neuron fires, you can't tell which of its concepts caused it without looking at the whole context. So you can't read what the model is doing just by looking at its neurons.
Steering a model runs into the same problem. Say you wanted to turn down something harmful, like writing malware, by damping the neurons involved. Those neurons also carry unrelated features, like ordinary Python debugging, so you would weaken those as well.
How researchers deal with it
Instead of reading individual neurons, researchers train a sparse autoencoder (SAE) on a layer's activations. The SAE maps the activations into a much wider space, with many more dimensions than the layer has neurons. It's trained to rebuild the original activations from that wider version while keeping only a few of its dimensions active at once. Because so few can be active at once, each dimension tends to settle on a single concept, and the mixed-up neurons get pulled apart into features that are mostly monosemantic.
Bricken et al. showed this working on a one-layer transformer, expanding its 512-neuron MLP layer into as many as 131,072 features. Templeton et al. (2024) then scaled it up to Claude 3 Sonnet, with SAEs of up to about 34 million features on the model's middle layer. The SAEs turned up features for the Golden Gate Bridge, for code with security vulnerabilities and for deception, among many others. Clamping the Golden Gate Bridge feature to ten times its maximum activation made the model start describing itself as the bridge (hilarious).
SAEs don't untangle everything, though. In Templeton et al.'s runs, the SAE's reconstruction explained at least 65% of the variance in the model's activations, so up to about a third went uncaptured. In the largest SAE, about 65% of the features were dead, meaning they never fired on a sample of 10 million tokens. Bricken et al. describe what they found as "relatively monosemantic" features, not perfectly clean ones.
Is it really superposition?
Nick Cammarata, one of the Zoom In authors, has recently pushed back on much of this in an essay called Neural Networks Are Already Clean. He argues that the cat and car neuron is really a V-shape neuron, which fires for pointed things and is pushed negative by round ones. He also says the "cars in superposition" circuit he found in InceptionV1 in 2019 was an error, and that he doesn't know of anyone finding a superposition circuit in a real network since. In his view, most polysemanticity comes from "recursive generalization", where a neuron grows from one concept into related ones, plus problems with architectures and tools that can be fixed. In preliminary audits of Qwen3-32B, his team estimates that about 90% of neuron "poles" (the inputs that drive a neuron strongly positive, or strongly negative) have one main job. He calls the essay a working hypothesis, but if it holds up, you could study many neurons directly instead of untangling them with an SAE first.
Wrapping up
If you've wondered why interpretability work talks about features and directions rather than neurons, polysemanticity is the reason. The refusal direction in my abliteration post is an example, since it's a direction in activation space that you find by comparing activations, not a single neuron you could switch off.
It's an exciting area of research, and there's still a lot to uncover.
References
- Olah, Cammarata, Schubert, Goh, Petrov & Carter. Zoom In: An Introduction to Circuits. Distill, 2020.
- Elhage, Hume, Olsson, Schiefer et al. Toy Models of Superposition. Transformer Circuits Thread, 2022.
- Bricken, Templeton, Batson, Chen, Jermyn et al. Towards Monosemanticity: Decomposing Language Models With Dictionary Learning. Transformer Circuits Thread, 2023.
- Templeton, Conerly et al. Scaling Monosemanticity: Extracting Interpretable Features from Claude 3 Sonnet. Transformer Circuits Thread, 2024.
- Cammarata. Neural Networks Are Already Clean. Goodfire.