Person
François Chollet
François Chollet is a French software engineer and AI researcher, best known as the creator of Keras, a widely used open-source library for building neural networks, and as the designer of ARC-AGI, a benchmark of novel visual-reasoning puzzles meant to test genuine reasoning rather than memorised knowledge. He argues that scaling up today's models is not by itself a path to general intelligence — a benchmark he built specifically to resist the pattern-matching at which large models excel. When OpenAI's o3 model posted a jump in ARC-AGI scores in late 2024, he called it significant while noting the model still "fails on very easy tasks," and his ARC Prize Foundation released harder successor benchmarks to keep the distinction sharp.
Appears alongside
Featured in threads
Tracks
- Benchmarks & progress 4
- Models & capabilities 1
ARC Prize Foundation launches ARC-AGI-3
Humans scored 100% and frontier AI scored 0.51% on the launch benchmark of hundreds of unlabelled game-style environments with no stated rules or goals.
Benchmarks & progress
o3 posts a breakthrough score on ARC-AGI
A low-compute configuration scored 75.7%, roughly matching the ARC Prize's human-performance threshold, at about $26 per task against roughly $5 for a human solver.
Benchmarks & progress · Models & capabilities
ARC Prize 2024 winners and technical report published
The top score rose from 33% to 55.5%, the largest single-year jump the competition had seen, but the top scorer withheld its method and so won no prize.
Benchmarks & progress
Announcing ARC Prize 2024
The best public score on ARC-AGI stood at 34%, up from 20% when Chollet introduced the benchmark five years earlier, still well below typical human performance.
Benchmarks & progress
Also mentioned in 2 entries
Referenced in passing — François Chollet isn't the main subject of these.