OpenAI publishes research on sparse circuits for interpretability

OpenAI published research on training deliberately sparse neural networks to make their internal computations easier for humans to follow. Standard transformers pack many overlapping concepts into the same weights and neurons, a phenomenon researchers call superposition, which makes it hard to trace why a model produced a given output. OpenAI’s approach instead penalised models during training until roughly one in a thousand weights remained non-zero, and restricted how many neuron outputs could stay active for any given input.

On curated tasks — such as counting matched brackets, detecting quotation marks, or recognising loops in short Python programs — the resulting circuits were small enough that researchers could isolate the specific handful of weights responsible and describe what each was doing, something not generally possible in dense models of comparable size. OpenAI reported that raw capability still scaled with a model’s total size, while interpretability tracked the count of non-zero weights specifically, meaning the two properties could in principle be tuned somewhat independently rather than always trading off one-for-one.

The models used were far smaller and more narrowly tested than production systems like GPT-5, and the technique’s applicability to frontier-scale models with open-ended tasks remains unproven. The work sits alongside other mechanistic-interpretability approaches, including sparse autoencoders applied after training, as an attempt to give researchers tools to audit a model’s reasoning directly rather than relying solely on its stated explanations — a capability labs have cited as relevant to detecting deceptive or misaligned behaviour before it causes harm.