How It Works
Scaling Laws
Empirical formulas describing how a model’s performance improves predictably as you increase its size, training data, and compute — used to plan how large to build a model before training it.
Origin
Jared Kaplan, Sam McCandlish, and colleagues at OpenAI, 2020.