Benchmarks · Reasoning & problem-solving
AGIEval
Whether a model can pass the same standardised human exams — college entrance tests, law school admissions, bar exams, maths competitions — that are used to select and qualify people, rather than a bespoke academic test built only for machines.
Microsoft Research (Zhong, Duan, Chen et al.)Released 13 April 2023Saturated
AGIEval took a different approach to evaluation than most benchmarks on this list: instead of writing new questions, Microsoft Research assembled 20 tasks straight from exams that already select and qualify people — the SAT, the LSAT, China’s Gaokao college entrance exam, bar qualification tests, maths competitions — and scored models against them exactly as a human candidate would be scored. The pitch was that passing a real standardised exam is a more legible claim about capability than topping a bespoke academic leaderboard.
The 2023 paper’s headline finding was that GPT-4 could already exceed average human performance on some of those exams: 95% on SAT Maths and 92.5% on the English-comprehension section of the Gaokao, both above the typical human test-taker. That result was uneven across the full task set rather than universal, and later evaluation by AGIEval’s own maintainers found GPT-4o averaging a more modest 62.3% across all 20 tasks in a zero-shot setting — a reminder that a strong result on one exam within a suite does not automatically generalise to the rest.
AGIEval has not spawned the kind of harder-successor arms race that MMLU or ARC-AGI did; instead it settled into use as one component of broader multi-benchmark comparison tables, valued for testing against real-world credentialing exams rather than academic proxies, but rarely cited as a standalone headline number in a model release.
The set
20 tasks (18 multiple-choice, two cloze) drawn from real-world exams including the SAT, LSAT, China's Gaokao college entrance exam, bar qualification tests and maths competitions, scored against the same passing standards used for human test-takers.
Example
SAT-en passage: 'Akira came directly, breaking all tradition. Was that it? Had he followed form—had he asked his mother to speak to his father to approach a go-between—would Chie have been more receptive? … "Depending on your response, I may stay in Japan. I've come to ask for Naomi's hand."' Question: 'Which choice best describes what happens in the passage?' (A) One character argues with another character who intrudes on her home. (B) One character receives a surprising request from another character. (C) One character reminisces about choices she has made over the years. (D) One character criticizes another character for pursuing an unexpected course of action. (Answer: B)github.com
Where it stands
GPT-4 exceeded average human performance on several component exams within the paper that introduced the benchmark; by the GPT-4o generation, average scores across all 20 tasks were still well short of ceiling, and AGIEval is now mainly cited as one line in broader evaluation tables rather than a standalone headline result.
How the top score changed hands
- April 2023GPT-495% (SAT Math) / 92.5% (Gaokao English)GPT-4 exceeded average human test-taker performance on several individual exams in the paper's own evaluation, though not on every task.
- May 2024GPT-4o62.3% (all 20 tasks, zero-shot)
Current best: GPT-4o — 71.4% (English tasks, few-shot) 62.3% averaged across all 20 tasks in a zero-shot setting, per the AGIEval GitHub repository's own evaluation table; later frontier models may score higher but are not documented there.