Anthropic publishes 'A Mathematical Framework for Transformer Circuits'

Anthropic’s interpretability team, led by Chris Olah, launched the Transformer Circuits Thread with a paper attempting to reverse-engineer transformer language models by studying them in their simplest possible forms and building up in complexity, rather than analysing full-scale production models directly.

Working with attention-only transformers of up to two layers — stripped of the multi-layer perceptron blocks present in real language models, to isolate the attention mechanism — the team showed that a zero-layer transformer’s behaviour could be read directly off its weights as a bigram statistics table, and that one-layer attention-only models functioned as an interpretable ensemble of bigram and “skip-trigram” predictions. The more consequential finding concerned two-layer models: composing attention heads across layers produced “induction heads,” a circuit that let the model complete a repeated pattern by copying whatever token had followed a matching earlier occurrence in the same context. The paper argued induction heads were a plausible mechanistic building block for in-context learning — the ability, central to how GPT-3 and its successors used a prompt’s examples without any weight updates, that Anthropic’s own founders had helped document.

The paper was explicit about its limits: the architectures studied were far simpler than any deployed model, and the authors described the work as early steps rather than a general method. Its significance lay in establishing a research programme rather than in the results themselves — mechanistic interpretability, aimed at explaining model behaviour in terms of specific, human-understandable computational circuits rather than only input-output correlations, became one of Anthropic’s core safety bets, and the Transformer Circuits Thread continued publishing findings, including further work explicitly building on induction heads, in the years that followed.