Hugging Face releases SmolLM
The largest variant, 1.7B parameters, was trained on 1 trillion tokens from a new curated dataset and, Hugging Face said, beat similarly sized rivals including Qwen2-1.5B.
- Open weights & ecosystem
- Models & capabilities
- Minor
Hugging Face released SmolLM, a family of small open language models in three sizes — 135M, 360M and 1.7B parameters — aimed at running on phones and other resource-constrained devices rather than servers. The 360M model, for instance, needed roughly 1.5GB of memory to run.
The models were trained on SmolLM-Corpus, a newly curated dataset combining Cosmopedia v2 (28 billion tokens of synthetic educational content), FineWeb-Edu (220 billion tokens of filtered educational web text) and a Python code dataset, on the premise that data quality mattered more than raw scale at small parameter counts. Training volume ranged from 600 billion tokens for the smaller models to 1 trillion for the 1.7B version. Hugging Face reported that each SmolLM size outperformed comparably sized competitors on its evaluation suite — the 1.7B model reportedly ahead of Microsoft’s Phi-1.5 and Alibaba’s Qwen2-1.5B.
The release reflected a growing 2024 trend of labs optimising for capability-per-parameter rather than absolute scale, aimed at on-device and edge deployment where cost, latency and memory footprint mattered more than matching frontier-model performance.