Artificial Analysis Coding Agent Index

Artificial Analysis’s Coding Agent Index asks a narrower question than its general Intelligence Index: not how capable a model is in the abstract, but how well a specific agent setup — a model wired into a coding harness such as Claude Code, Codex or Cursor CLI — gets through real software work. Results are reported by that model-and-harness pairing rather than by model alone, because the same underlying model can score differently depending on which tool-calling scaffold it runs inside.

The index combines three benchmarks in equal measure: DeepSWE’s long-horizon engineering tasks, Terminal-Bench v2’s agentic terminal use, and Scale AI’s SWE-Atlas-QnA repository question-answering set, each scored on the first-attempt pass rate averaged over three tries. Alongside the composite score, Artificial Analysis publishes cost, token usage and wall-clock time per task, letting a reader weigh a faster or cheaper agent against a marginally more capable one rather than reading capability alone. When OpenAI released GPT-5.6 in July 2026, it cited a score of 80 for its Sol model running at maximum reasoning effort, ahead of Anthropic’s Claude Fable 5 on the same index at the time, while using fewer tokens and less compute per task.

Because it scores agent-and-harness combinations rather than static model outputs, and because its component benchmarks are comparatively new and still being iterated — the index stood at version 1.3 as of August 2026 — it functions less as a settled reference than as a running snapshot of which combination of model and tooling currently gets real coding work done fastest and cheapest.