Organisation
EleutherAI
A research collective turned non-profit that pioneered openly released language models (GPT-Neo, GPT-J) and open training corpora (The Pile).
EleutherAI began in 2020 as a Discord server where researchers and hobbyists discussed GPT-3 after OpenAI declined to release its weights, and turned into a volunteer collective with the explicit aim of training and openly releasing a comparable model. Working without institutional backing, it delivered the GPT-Neo and GPT-J model families and The Pile, an open training corpus built because no open equivalent to OpenAI's training data existed. Its founding is often cited as the point at which open weights became an organised counter-current to the frontier labs, rather than an occasional side release. Now a non-profit, it has broadened into interpretability and alignment research and continued work on openly licensed datasets, including the copyright-conscious Common Pile released in 2025.
- Category
- Open source & ecosystem
- Founded
- 2020
- Key people
- Stella Biderman
Appears alongside
Featured in threads
Tracks
- Open weights & ecosystem 7
- Models & capabilities 3
- Ideas & essays 1
EleutherAI releases the Common Pile v0.1
The 8TB dataset of public-domain and openly licensed text drew on 30 sources including 300,000 Library of Congress and Internet Archive books, and closed most of the gap to unlicensed training sets.
Open weights & ecosystem
EleutherAI releases Pythia model suite
Every model in the eight-size suite was trained on identical data in the same order, with 154 saved checkpoints each, to let researchers study training dynamics directly.
Open weights & ecosystem
EleutherAI announces GPT-NeoX-20B
EleutherAI released GPT-NeoX-20B, a 20-billion-parameter open model, its largest dense model to date and freely available via GitHub.
Open weights & ecosystem · Models & capabilities
EleutherAI releases GPT-J-6B
At 6 billion parameters it scored close to OpenAI's similarly sized GPT-3 model on the LAMBADA benchmark, and its weights were downloadable under Apache 2.0.
Open weights & ecosystem · Models & capabilities
EleutherAI releases GPT-Neo
The 1.3B and 2.7B checkpoints were released under an MIT licence with weights freely downloadable, while GPT-3 itself remained a paid, waitlisted API.
Open weights & ecosystem · Models & capabilities
EleutherAI releases The Pile
The 825GiB corpus combined 22 curated sources rather than raw web scrapes, and models trained on it outperformed Common Crawl-only baselines on academic text.
Open weights & ecosystem · Ideas & essays
EleutherAI forms to build open language models
A Discord server for discussing GPT-3 became a volunteer collective aiming to train and openly release a comparable model itself.
Open weights & ecosystem
Also mentioned in 1 entry
Referenced in passing — EleutherAI isn't the main subject of these.