Fetching the paper…
Reading the bibliography…
Recently a number of empirical "universal" scaling law papers have been published, most notably by OpenAI.
On Estimating the Probability of Discovering a New Species
Anne Chao · 1981
Earlier work this paper cites.
Optimal nonlinear approximation
Ronald A. DeVore, Ralph Howard, and Charles Micchelli · 1989
Earlier work this paper cites.
Neural Networks for Optimal Approximation of Smooth and Analytic Functions
H. N. Mhaskar · 1996
Earlier work this paper cites.
Approximation theory of the MLP model in neural networks
Allan Pinkus · 1999
Earlier work this paper cites.
A Bayesian review of the Poisson-Dirichlet process
Wray Buntine and Marcus Hutter · 2010
Earlier work this paper cites.
Convex Optimization: Algorithms and Complexity
Sébastien Bubeck · 2015
Earlier work this paper cites.
Introduction to Online Convex Optimization
Elad Hazan · 2016
Cited alongside, same era.
Deep Learning Scaling is Predictable, Empirically
Joel Hestness, Sharan Narang, Newsha Ardalani, Gregory Diamos, Heewoo Jun, Hassan Kianinejad, Md Mostofa Ali Patwary, Yang Yang, and Yanqi Zhou · 2017
Cited alongside, same era.
Overfitting or perfect fitting? Risk bounds for classification and regression rules that interpolate
Mikhail Belkin, Daniel Hsu, and Partha Mitra · 2018
Cited alongside, same era.
Reconciling modern machine-learning practice and the classical bias–variance trade-off
Mikhail Belkin, Daniel Hsu, Siyuan Ma, and Soumik Mandal · 2019
Cited alongside, same era.
A Constructive Prediction of the Generalization Error Across Scales
Jonathan S. Rosenfeld, Amir Rosenfeld, Yonatan Belinkov, and Nir Shavit · 2019
Cited alongside, same era.
Scaling laws of recovering Bernoulli, November 2020
Kyunghyun Cho · 2020
Later among the works it cites.
Scaling Laws for Autoregressive Generative Modeling
Tom Henighan, Jared Kaplan, Mor Katz, Mark Chen, Christopher Hesse, Jacob Jackson, Heewoo Jun, Tom B. Brown, Prafulla Dhariwal, Scott Gray, Chris Hallacy, Benjamin Mann, Alec Radford, Aditya Ramesh, Nick Ryder, Daniel M. Ziegler, John Schulman, Dario Amodei, and Sam McCandlish · 2020
Later among the works it cites.
Scaling Laws for Neural Language Models
Jared Kaplan, Sam McCandlish, Tom Henighan, Tom B. Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, and Dario Amodei · 2020
Later among the works it cites.
A Neural Scaling Law from the Dimension of the Data Manifold
Utkarsh Sharma and Jared Kaplan · 2020
Later among the works it cites.
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Danny Hernandez, Jared Kaplan, Tom Henighan, and Sam McCandlish · 2021
Closest in time.