GPQA

GPQA’s name is a description of what it was built to defeat: “Google-proof” questions, graduate-level enough that looking them up doesn’t help. Researchers published it in November 2023 with 448 multiple-choice questions in biology, physics and chemistry, each written and checked by a PhD holder in that exact specialism. The paper’s baselines made the point sharply: domain experts scored 65% on questions inside their own field, but skilled non-experts given unrestricted web access and more than half an hour per question managed only 34% — barely above chance. GPT-4, the best model tested, reached 39%.

The gap the benchmark was designed to expose — between people who could look things up and people who actually understood the material — turned out to be exactly the gap reasoning-focused models started closing fastest. Scores climbed quickly through 2024: Claude 3.5 Sonnet improved on its predecessor, and by December 2024 OpenAI’s o3 reported 87.7% on the harder 198-question “Diamond” subset that had become the standard citation — well past the original PhD-expert baseline.

That baseline stopped being much of a ceiling. By late 2025, Gemini 3 Pro reported 91.9% on GPQA Diamond, rising to 93.8% in its higher-effort Deep Think mode, and GPQA had joined the small set of benchmarks — alongside MMLU before it — that most frontier labs cite in every release even as scores approach saturation. It remains one of the most consistently reported science benchmarks precisely because its original difficulty took so long to fall.