AISI, Anthropic and Alan Turing Institute find just 250 documents can backdoor an LLM regardless of model size

Researchers from the UK AI Security Institute, Anthropic and the Alan Turing Institute reported that as few as 250 malicious documents inserted into a training corpus could implant a backdoor in a large language model, and that this held regardless of the model’s overall size or the volume of clean data it was trained on. The team trained 72 models ranging from 600 million to 13 billion parameters, testing poisoning levels of 100, 250 and 500 documents, and described it as the largest data-poisoning study conducted to date.

The finding challenged a working assumption in the field: that poisoning attacks scale with the proportion of a training set an attacker controls, meaning larger, more heavily trained models would need proportionally more corrupted data to compromise — and would therefore be harder to attack in practice as datasets grew. The study instead found that a fixed, small number of documents was sufficient across the size range tested, so a 13-billion-parameter model trained on far more data than a 600-million-parameter one was no harder to backdoor with the same absolute quantity of poisoned text.

The researchers were explicit about the limits of the result. The backdoor tested was narrow — designed to trigger the model into producing gibberish output on a specific phrase — a low-stakes behaviour chosen for its measurability rather than because it resembled a realistic threat. They said it remained uncertain whether the finding would generalise to larger frontier models or to more consequential backdoors, such as ones that inserted exploitable code or bypassed safety guardrails. Even with that caveat, the result reframed data poisoning as a threat whose feasibility does not fall as a lab scales up its training data, which had been the implicit basis for treating the risk as diminishing with model size.