LMArena

Most benchmarks score a model against a fixed answer key. LMArena, launched by LMSYS researchers at UC Berkeley as Chatbot Arena in May 2023, asks a different question: given two anonymous replies to the same prompt, which one does a person actually prefer? Visitors vote blind, the identities are revealed afterwards, and the votes accumulate into a running Elo-style rating rather than a one-off score — closer to how the models are used in practice, but harder to audit than a test with a right answer.

The approach made LMArena one of the most closely watched public rankings in the field, cited in launch announcements from every major lab as the votes piled into the millions. That prominence also made it a target. In April 2025 it emerged that Meta had submitted a specially tuned Llama 4 Maverick variant that outperformed the model it actually shipped, and weeks later “The Leaderboard Illusion” argued that undisclosed private pre-release testing structurally favoured well-resourced labs. LMArena disputed several of the paper’s figures but adopted disclosure and provisional-scoring policies in response.

Leadership has continued to change hands quickly since — Google’s Gemini 2.5 Pro and later Gemini 3 Pro both took the top spot on release, and by August 2026 the text arena, now rebranded simply “Arena,” had logged more than 7.7 million votes across 389 models, with Anthropic, Google and OpenAI systems clustered within a few Elo points at the top. It remains a widely cited signal of how models are received in open-ended use, but after 2025’s disputes it is generally read alongside fixed benchmarks rather than in place of them.