2021: foundations, and the tools to build on them

If 2020 proved the idea, 2021 was about building on it. The year opened with OpenAI’s DALL·E and CLIP, early gestures towards generating and understanding images from text, and it filled out with tools that put large models to practical work. GitHub’s Copilot, powered by OpenAI’s Codex, began suggesting code to programmers as they typed — the first time many people used an AI system every day without thinking of it as one. AlphaFold 2 was published in full and open-sourced, and the accompanying database of predicted protein structures put a genuinely useful scientific tool in the hands of researchers worldwide. Stanford gathered the whole phenomenon under a single label, “foundation models”, capturing the idea that a handful of large, general systems were becoming the base layer everything else was built on. Newer labs formed to pursue the same frontier from different angles, Anthropic among them, and quieter advances — cheaper fine-tuning, the first serious attempts to see inside a model’s workings — laid groundwork that mattered more than the headlines suggested.

The counter-arguments matured alongside the tools. The paper “On the Dangers of Stochastic Parrots” set out, carefully, the case that a bigger language model is a more convincing mimic rather than a better thinker, and that the costs of building one — environmental, financial, social — deserved more scrutiny than they were getting. In April the European Commission proposed the AI Act, the first serious attempt anywhere to write binding rules for the technology, though it would be years before anything took effect. Copilot, meanwhile, drew immediate questions about training on publicly posted code without asking — an early version of the copyright fight that would grow much larger. For most people, all of this was still happening somewhere off to the side.