"Let's think step by step" elicits zero-shot reasoning

Four months after chain-of-thought prompting was described, Takeshi Kojima, Shixiang Shane Gu and colleagues at the University of Tokyo and Google Research showed that the worked examples were not necessary. Simply appending the phrase “Let’s think step by step” to a question, with no demonstrations of reasoning at all, produced much of the same effect — a technique the authors called Zero-shot-CoT.

The reported gains were large. On MultiArith, a benchmark of arithmetic word problems, accuracy on a GPT-3-class model rose from 17.7% to 78.7%; on GSM8K, grade-school math problems, from 10.4% to 40.7%. The prompt worked across arithmetic, symbolic and logical reasoning tasks, and the authors found it was robust to the exact wording — several similar instructions produced comparable improvements. As with the few-shot version, the effect appeared only in sufficiently large models.

The result sharpened the claim of the earlier work. Chain-of-thought had shown that reasoning could be elicited by prompting rather than training; the zero-shot version showed that even the eight hand-written exemplars were dispensable, and that a single fixed instruction could unlock the same latent capability. That a phrase this simple had such an effect became one of the most cited and most imitated findings in prompting, and “let’s think step by step” entered the field’s working vocabulary. It reinforced the direction that later reasoning-focused models would take further: get the model to externalise its intermediate steps before committing to an answer.