Model

megatron-turing-nlg-530b

1 entry · 11 October 2021

Appears alongside

Featured in threads

Tracks

  • Models & capabilities 1
  • Compute & infrastructure 1

  1. Microsoft and NVIDIA train Megatron-Turing NLG at 530 billion parameters

    The largest dense language model publicly described at the time, and a demonstration of multi-thousand-GPU training.

    Models & capabilities · Compute & infrastructure