Meta releases Llama 3.1 405B

Meta released Llama 3.1, including a 405-billion-parameter model it described as its most capable to date and the largest openly available foundation model at the time. The 405B model was trained on more than 15 trillion tokens using 16,000 Nvidia H100 GPUs, with a 128,000-token context window, and shipped alongside updated 8B and 70B versions. Meta reported that on its evaluations the 405B model was competitive with GPT-4, GPT-4o and Claude 3.5 Sonnet across a range of tasks — the first time an open-weight model’s benchmark scores put it in the same conversation as the closed frontier rather than a tier behind it, though independent testers found the gap narrower on some tasks than others and wider on some reasoning benchmarks.

Mark Zuckerberg published an accompanying essay, “Open Source AI Is the Path Forward,” arguing that open-weight release served developers, Meta and society by preventing AI capability from concentrating in a small number of companies, and framing openness as a security and competitiveness argument rather than only a philosophical one. The release remained “open-weight” rather than fully open: Meta’s licence carried restrictions, including a requirement that companies with more than 700 million monthly active users obtain a separate licence from Meta, and did not release training data or code.

The models were made available immediately through Meta’s own site, Hugging Face and more than twenty-five partner platforms including AWS, Nvidia and Google Cloud. Llama 3.1 405B became the reference point cited on both sides of the argument over whether open-weight release was a viable competitive strategy against closed frontier labs or primarily a way for a company without a leading consumer AI product to commoditise its rivals’ advantage.