AI2 releases OLMoTrace for tracing model outputs to training data
The tool highlights spans of a model's output that match its training data verbatim and links to the source documents, using an indexed search algorithm the researchers said scaled to trillions of tokens.
- Open weights & ecosystem
- Safety & alignment
- Colour
The Allen Institute for AI released OLMoTrace, a feature added to its AI2 Playground that traces a language model’s outputs back to matching passages in the full training corpus in real time. The tool identifies long, distinctive spans of text in a model’s generated response that also appear verbatim in the training data, ranks candidate matches, and surfaces up to ten source documents per span so a user can inspect where a specific phrase or claim may have come from.
AI2 said the underlying search relied on an indexed dataset technique, infini-gram, paired with a parallel algorithm designed to make document lookup practical at the scale of a multi-trillion-token corpus rather than only on small samples. At launch it worked with AI2’s own fully open OLMo 2 and OLMoE models, whose training data AI2 could actually disclose — something not possible for closed frontier models whose training sets are undisclosed.
OLMoTrace was a small release but a pointed one: it demonstrated a capability — inspecting the relationship between a specific output and specific training documents — that depended entirely on having open training data to trace against, reinforcing AI2’s argument that full openness enabled scrutiny closed labs could not offer.