EleutherAI releases the Common Pile v0.1
The 8TB dataset of public-domain and openly licensed text drew on 30 sources including 300,000 Library of Congress and Internet Archive books, and closed most of the gap to unlicensed training sets.
Open weights & ecosystem