Databricks releases Dolly 2.0

Databricks released Dolly 2.0, a 12-billion-parameter language model built on EleutherAI’s Pythia architecture and fine-tuned for instruction-following. The company described it as the first openly licensed model of its kind where every component — weights, training code and the instruction dataset itself — was cleared for unrestricted commercial use.

That last point was the release’s distinguishing claim. Earlier open instruction-tuned models such as Stanford’s Alpaca had been fine-tuned on responses generated by ChatGPT, which OpenAI’s terms of service prohibit from being used to train a competing commercial product. Databricks instead crowdsourced a new dataset, databricks-dolly-15k, containing 15,000 instruction-response pairs written by more than 5,000 of its own employees across categories including open Q&A, summarisation, brainstorming and creative writing, and released it under a Creative Commons licence permitting commercial use.

Coverage at the time noted the model’s limitations alongside the licensing achievement: TechCrunch found it gave inaccurate and stereotyped answers to a prompt about gender-typical jobs, illustrating that an open, commercially clear licence did not imply a well-behaved model. Dolly 2.0 followed Dolly 1.0, released by Databricks less than three weeks earlier, and arrived the same week as Cerebras’s open model family, part of a spring 2023 wave of attempts to build an open alternative to closed chatbot APIs. Databricks acquired MosaicML two months later, folding both companies’ open-model work into one team.