OpenAI publishes research on why language models hallucinate

OpenAI published a paper, written with a Georgia Tech co-author, arguing that hallucination in large language models is not simply a byproduct of imperfect training data but a predictable consequence of how models are trained and evaluated. Authored by Adam Tauman Kalai, Ofir Nachum and Edwin Zhang of OpenAI together with Santosh Vempala of Georgia Tech, the paper framed the phenomenon in statistical terms: language models learn to produce plausible continuations under uncertainty much as a student guesses on an exam, because standard scoring rewards a confident guess with some chance of being right over an admission of “I don’t know,” which guarantees no credit.

The paper’s central claim was that most benchmarks used to evaluate language models score only binary accuracy, so a model that always guesses will outperform one that abstains when uncertain, even though the guessing model is less trustworthy in practice. This creates an incentive, baked into the leaderboards labs compete on, to suppress calibrated uncertainty rather than express it. The authors argued the fix has to be socio-technical rather than purely architectural: changing how benchmarks are scored, to reward calibrated abstention and penalise confident error, rather than relying solely on training interventions to reduce hallucination directly.

The paper did not claim to eliminate hallucination or offer a new technique that measurably reduced it in a shipped model; it was a diagnostic and evaluation-design argument rather than a capabilities announcement. It drew attention for locating the incentive problem in benchmark design, a target that implicated the wider research community’s evaluation culture rather than any single company’s training pipeline, and added to a body of 2025 work examining why hallucination rates had not fallen as quickly as other capability metrics had improved.