GPT-3 shows few-shot learning at scale

OpenAI's 175B-parameter GPT-3 performs new tasks from a handful of examples in the prompt, without fine-tuning.

What happened and why it matters

The GPT-3 paper demonstrated that scaling a plain transformer language model by ~100x over GPT-2 produced in-context learning: translation, arithmetic, question answering and code from a few examples. It made scaling laws the dominant strategy of the field and started the foundation-model era.

Sources

• Language Models are Few-Shot Learners ↗ arXiv · 28 May 2020