Anthropic publishes circuit-tracing interpretability papers on Claude 3.5 Haiku

Anthropic published “On the Biology of a Large Language Model,” alongside a companion methods paper, “Circuit Tracing: Revealing Computational Graphs in Language Models,” describing a technique for tracing how Claude 3.5 Haiku arrives at its outputs. The method replaces the model’s densely entangled internal neurons with roughly 30 million more interpretable features, using “cross-layer transcoders,” then builds attribution graphs that trace the causal path from a prompt through intermediate computations to the eventual output.

The resulting case studies were the most concrete evidence to date that a production language model does something closer to internal reasoning than pure next-token prediction. When asked for the capital of the state containing Dallas, the model was shown to activate an intermediate “Texas” representation before producing “Austin” — verified by swapping that feature for a different state and watching the answer change accordingly. When writing rhyming couplets, the model appeared to select a candidate end-word for a line before composing the words leading up to it, evidence of forward planning rather than word-by-word improvisation. Other case studies covered multilingual processing — a mix of language-specific routing and shared, language-independent semantic features, with English acting as a mechanistically privileged default — arithmetic circuits that generalised across unrelated contexts, and the internal mechanics of refusals and of at least one jailbreak, which worked by delaying the model’s recognition that a request it had already begun answering was harmful.

Anthropic was candid about the method’s limits: attribution graphs are hypotheses that need separate intervention experiments to confirm, the simplified “replacement” model used for tracing does not fully reproduce the original network’s behaviour, and the technique produced satisfying explanations for only about a quarter of the prompts tried, drawn from a sample biased toward cases the method could handle. Even with those caveats, the work was read as a significant step for mechanistic interpretability — showing internal structure in a model close to production scale rather than the small, purpose-built networks the field had mostly studied before, and adding evidence to a running argument about whether a model’s stated reasoning reflects what it actually computed.