Anthropic publishes 'Towards Monosemanticity'

Anthropic’s interpretability team, led by Chris Olah, published “Towards Monosemanticity,” showing that a technique called sparse dictionary learning could pull thousands of human-interpretable “features” out of a small language model’s neurons — a step toward reading what a model is actually representing internally rather than treating it as a black box.

The problem the paper addressed is polysemanticity: individual neurons in a transformer typically respond to many unrelated concepts at once, because the network is packing more ideas into its activations than it has neurons to hold them (a phenomenon Anthropic calls superposition). The authors trained a sparse autoencoder on the activations of a one-layer, 512-neuron transformer and used it to decompose those activations into a much larger set of sparse, largely independent directions. The result was more than 4,000 features, each one considerably easier for a human rater to interpret than the raw neurons had been — features corresponding to specific concepts such as Arabic script, DNA sequences, legal boilerplate, HTTP requests and Hebrew text, several of which could be shown to causally drive the model’s behaviour when directly activated.

The authors were explicit about the limits of the demonstration. The model studied was tiny by the standards of the field, and they described the central open question as scaling the method to “frontier models which are many times larger and substantially more complicated” — characterising the remaining challenge as chiefly engineering rather than a further scientific unknown.

That characterisation was tested seven months later. In May 2024 the same team published “Scaling Monosemanticity,” applying the technique to Claude 3 Sonnet, a production-scale model, and finding millions of interpretable features including some tied to safety-relevant behaviour. The October 2023 paper is generally treated as the method’s proof of concept — the point at which mechanistic interpretability, previously confined to toy models, acquired a technique its practitioners believed could plausibly scale.