Model

gpt-3

14 entries · May 2020 – May 2022

GPT-3 was OpenAI's 2020 language model with 175 billion parameters, roughly ten times larger than any dense model publicly described before it. Its central finding was that a model this large could pick up new tasks from a few examples placed in its prompt, without any retraining — the "few-shot" learning that made scale itself look like the route to more capability. OpenAI released it not as downloadable weights but through a paid, invitation-only API, setting the pattern of metered access that shaped the years that followed.

Appears alongside

Featured in threads

Tracks

  • Models & capabilities 6
  • Ideas & essays 4
  • Money & business 4
  • Benchmarks & progress 3
  • Safety & alignment 2
  • Culture & impact 2
  • Labs & people 1
  • Open weights & ecosystem 1
  • Compute & infrastructure 1

  1. "Let's think step by step" elicits zero-shot reasoning

    A single prompt phrase, with no worked examples, lifted GSM8K accuracy from 10.4% to 40.7% — chain-of-thought without the exemplars.

    Ideas & essays · Benchmarks & progress

  2. OpenAI ships InstructGPT and makes RLHF the default

    Models fine-tuned on human preference data were preferred to a model a hundred times larger, reframing alignment as a product feature.

    Models & capabilities · Safety & alignment · Ideas & essays

  3. OpenAI publishes WebGPT, a model that browses the web to answer questions

    Answers preferred to Reddit's top-voted responses 69% of the time in blind comparison, but the model still fell short of human accuracy on TruthfulQA.

    Models & capabilities

  4. OpenAI removes the GPT-3 waitlist

    General availability in supported countries turned the API from a curated experiment into a commodity developers could just buy, though several countries were excluded.

    Models & capabilities · Money & business

  5. TruthfulQA measures whether models repeat human falsehoods

    On 817 questions designed to elicit common misconceptions, the best model tested was truthful only 58% of the time against 94% for humans, and larger models scored worse.

    Benchmarks & progress · Safety & alignment

  6. Microsoft takes an exclusive licence to GPT-3

    Microsoft gained access to the underlying model and source code for its own products; the public API, through which OpenAI still served other customers, was unaffected.

    Money & business · Labs & people · Models & capabilities

  7. The Guardian publishes an op-ed written by GPT-3

    Editors ran the model eight times on the same prompt and spliced the best passages together, a process disclosed in a footnote that critics said undercut the framing.

    Culture & impact

  8. Hendrycks et al. publish the MMLU benchmark

    15,908-question, 57-subject multiple-choice benchmark spanning elementary to professional level; GPT-3 improved on random chance by roughly 20 points on average.

    Benchmarks & progress

  9. GPT-3 demos spread beyond the research community

    Developer Sharif Shameem's demo turning plain-English descriptions into working webpage code was among the clips that drew attention beyond NLP researchers.

    Culture & impact

  10. EleutherAI forms to build open language models

    A Discord server for discussing GPT-3 became a volunteer collective aiming to train and openly release a comparable model itself.

    Open weights & ecosystem

  11. OpenAI opens the GPT-3 API in private beta

    Access required a waitlisted application rather than a download, and OpenAI said the model itself would stay unpublished.

    Models & capabilities · Money & business

  12. OpenAI publishes the GPT-3 paper

    A 175-billion-parameter language model that learned new tasks from examples in its prompt, without any weight updates.

    Models & capabilities · Ideas & essays

  13. Gwern publishes 'The Scaling Hypothesis'

    Gwern's essay, written around GPT-3's release, argued scale alone was producing qualitatively new abilities and became a widely cited framing for the scaling-hypothesis argument.

    Ideas & essays

  14. Microsoft reveals the supercomputer it built for OpenAI

    Announced at Build, a top-five ranked machine dedicated to a single customer — an early signal of AI's compute economics.

    Compute & infrastructure · Money & business

Also mentioned in 1 entry

Referenced in passing — gpt-3 isn't the main subject of these.