TruthfulQA measures whether models repeat human falsehoods

Researchers Stephanie Lin and Owain Evans, of Oxford’s Future of Humanity Institute, and Jacob Hilton, of OpenAI, published TruthfulQA, a benchmark of 817 questions across 38 categories — including health, law, finance and politics — deliberately written so that some people would answer them falsely because of a common misconception. A model could not score well simply by imitating typical web text, since imitating typical text was exactly what produced the false answers.

Testing GPT-3, GPT-Neo/GPT-J, GPT-2 and a T5-based model, the authors found the best-performing model was truthful on 58% of questions, against 94% for a human baseline. The more striking finding was directional rather than absolute: within each model family, larger versions were generally less truthful than smaller ones, the opposite of the pattern typically seen on other language benchmarks, where performance improves with scale. The authors argued this was expected, not paradoxical — if a model’s false answers are learned from imitating popular misconceptions in its training data, then a larger model that imitates that data more faithfully will reproduce those misconceptions more reliably too.

The paper’s suggested remedy was to move away from imitation-based training objectives rather than to rely on further scaling. TruthfulQA became one of the standard evaluations cited whenever a new model’s release materials described work on reducing hallucination, and its central finding — that scale alone does not fix truthfulness, and can worsen it — became a recurring reference point in arguments that capability and reliability were not the same axis.