Built on

G. E. Moore, “Cramming more components onto integrated circuits,” Electronics , vol. 38, pp. 114–117, 1965
1965
Earlier work this paper cites.
R. H. Dennard, E. Gaensly, and F. Spreen, “Design of ion-implanted MOSFETs with very small gate lengths,” IEEE transactions on electron devices , vol. 21, no. 1, pp. 637–644, 1974
1974
Earlier work this paper cites.
R. N. Foster, Innovation: The Attacker’s Advantage . New York, NY: Summit Books, 1986
1986
Earlier work this paper cites.
J. Kaplan, S. McCandlish, T. Henighan, T. B. Brown, B. Chess, R. Child, S. Gray, A. Radford, J. Wu, and D. Amodei, “Scaling laws for neural language models,” arXiv preprint arXiv:2001.08361 , 2020. [Online]. Available: https://arxiv.org/abs/2001.08361
Original
2001
Earlier work this paper cites.
T. B. Brown, B. Mann, N. Ryder, M. Subbiah, J. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell et al. , “Language models are few-shot learners,” arXiv preprint arXiv:2005.14165 , 2020. [Online]. Available: https://arxiv.org/abs/2005.14165
Original
2005
Earlier work this paper cites.
J. Hoffmann, S. Borgeaud, A. Mensch, T. Cai, E. Rutherford, D. de Las Casas, L. M. Hendricks, J. Welbl, A. Clark, J. Bewersdorf et al. , “Training compute-optimal large language models,” arXiv preprint arXiv:2203.15556 , 2022. [Online]. Available: https://arxiv.org/abs/2203.15556
Original
2022
Earlier work this paper cites.
A. Clark, D. de Las Casas, A. Guy, A. Mensch, M. Paganini, J. Hoffmann, B. Damoc, B. Hechtman, T. Cai, S. Borgeaud et al. , “Unified scaling laws for routed language models,” in Proceedings of the International Conference on Machine Learning . PMLR, 2022, pp. 4057–4086. [Online]. Available: https://proceedings.mlr.press/v162/clark22a.html
2022
Earlier work this paper cites.
A. Komatsuzaki, J. Puigcerver, J. Lee-Thorp, C. Riquelme, B. Mustafa, J. Ainslie, Y. Tay, M. Dehghani, and N. Houlsby, “Sparse upcycling: Training mixture-of-experts from dense checkpoints,” arXiv preprint arXiv:2212.05055 , 2022. [Online]. Available: https://arxiv.org/abs/2212.05055
Original
2022
Earlier work this paper cites.
E. Zelikman, Y. Wu, J. Mu, and N. D. Goodman, “STaR: Bootstrapping reasoning with reasoning,” arXiv preprint arXiv:2203.14465 , 2022. [Online]. Available: https://arxiv.org/abs/2203.14465
Original
2022
Earlier work this paper cites.

Similar

C. Snell, J. Lee, K. Xu, and A. Kumar, “Scaling LLM test-time compute optimally can be more effective than scaling model parameters,” arXiv preprint arXiv:2408.03314 , 2023. [Online]. Available: https://arxiv.org/abs/2408.03314
Original
2023
Cited alongside, same era.
Reuters, “AI with reasoning power will be less predictable, Ilya Sutskever says,” Reuters Technology , 2024. [Online]. Available: https://www.reuters.com/technology/artificial-intelligence/ai-with-reasoning-power-will-be-less-predictable-ilya-sutskever-says-2024-12-14/
2024
Cited alongside, same era.
Business Insider, “OpenAI is reportedly struggling to improve its next big AI model. It’s a warning for the entire AI industry.” Business Insider , 2024. [Online]. Available: https://www.businessinsider.com/openai-orion-model-scaling-law-silicon-valley-chatgpt-2024-11
2024
Cited alongside, same era.
Time Magazine, “Has AI progress really slowed down?” Time , 2024. [Online]. Available: https://time.com/7178328/is-ai-progress-slowing-down/
2024

Then

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…

2025

The Race to Efficiency: A New Perspective on AI Scaling Laws

Lu, Chien-Ping

Understand

As large-scale AI models expand, training becomes costlier and sustaining progress grows harder.

Classical scaling laws (e.g., Kaplan et al.
(2020), Hoffmann et al.
(2022)) predict training loss from a static compute budget yet neglect time and efficiency, prompting the question: how can we balance ballooning GPU fleets with rapidly improving hardware and algorithms? We introduce the relative-loss equation, a time- and efficiency-aware framework that extends classical AI scaling laws.

Built on

G. E. Moore, “Cramming more components onto integrated circuits,” Electronics , vol. 38, pp. 114–117, 1965
1965
Earlier work this paper cites.
R. H. Dennard, E. Gaensly, and F. Spreen, “Design of ion-implanted MOSFETs with very small gate lengths,” IEEE transactions on electron devices , vol. 21, no. 1, pp. 637–644, 1974
1974
Earlier work this paper cites.
R. N. Foster, Innovation: The Attacker’s Advantage . New York, NY: Summit Books, 1986
1986
Earlier work this paper cites.
J. Kaplan, S. McCandlish, T. Henighan, T. B. Brown, B. Chess, R. Child, S. Gray, A. Radford, J. Wu, and D. Amodei, “Scaling laws for neural language models,” arXiv preprint arXiv:2001.08361 , 2020. [Online]. Available: https://arxiv.org/abs/2001.08361
Original
2001
Earlier work this paper cites.
T. B. Brown, B. Mann, N. Ryder, M. Subbiah, J. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell et al. , “Language models are few-shot learners,” arXiv preprint arXiv:2005.14165 , 2020. [Online]. Available: https://arxiv.org/abs/2005.14165
Original
2005
Earlier work this paper cites.
J. Hoffmann, S. Borgeaud, A. Mensch, T. Cai, E. Rutherford, D. de Las Casas, L. M. Hendricks, J. Welbl, A. Clark, J. Bewersdorf et al. , “Training compute-optimal large language models,” arXiv preprint arXiv:2203.15556 , 2022. [Online]. Available: https://arxiv.org/abs/2203.15556
Original
2022
Earlier work this paper cites.
A. Clark, D. de Las Casas, A. Guy, A. Mensch, M. Paganini, J. Hoffmann, B. Damoc, B. Hechtman, T. Cai, S. Borgeaud et al. , “Unified scaling laws for routed language models,” in Proceedings of the International Conference on Machine Learning . PMLR, 2022, pp. 4057–4086. [Online]. Available: https://proceedings.mlr.press/v162/clark22a.html
2022
Earlier work this paper cites.
A. Komatsuzaki, J. Puigcerver, J. Lee-Thorp, C. Riquelme, B. Mustafa, J. Ainslie, Y. Tay, M. Dehghani, and N. Houlsby, “Sparse upcycling: Training mixture-of-experts from dense checkpoints,” arXiv preprint arXiv:2212.05055 , 2022. [Online]. Available: https://arxiv.org/abs/2212.05055
Original
2022
Earlier work this paper cites.
E. Zelikman, Y. Wu, J. Mu, and N. D. Goodman, “STaR: Bootstrapping reasoning with reasoning,” arXiv preprint arXiv:2203.14465 , 2022. [Online]. Available: https://arxiv.org/abs/2203.14465
Original
2022
Earlier work this paper cites.

Similar

C. Snell, J. Lee, K. Xu, and A. Kumar, “Scaling LLM test-time compute optimally can be more effective than scaling model parameters,” arXiv preprint arXiv:2408.03314 , 2023. [Online]. Available: https://arxiv.org/abs/2408.03314
Original
2023
Cited alongside, same era.
Reuters, “AI with reasoning power will be less predictable, Ilya Sutskever says,” Reuters Technology , 2024. [Online]. Available: https://www.reuters.com/technology/artificial-intelligence/ai-with-reasoning-power-will-be-less-predictable-ilya-sutskever-says-2024-12-14/
2024
Cited alongside, same era.
Business Insider, “OpenAI is reportedly struggling to improve its next big AI model. It’s a warning for the entire AI industry.” Business Insider , 2024. [Online]. Available: https://www.businessinsider.com/openai-orion-model-scaling-law-silicon-valley-chatgpt-2024-11
2024
Cited alongside, same era.
Time Magazine, “Has AI progress really slowed down?” Time , 2024. [Online]. Available: https://time.com/7178328/is-ai-progress-slowing-down/
2024

Then

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Cited alongside, same era.

Business Insider, “NVIDIA boss Jensen Huang predicts computing power will increase a ’millionfold’ in a decade,” Business Insider , 2024. [Online]. Available: https://www.businessinsider.com/nvidia-jensen-huang-predicts-increase-computing-power-ai-scaling-2024-11

2024

Cited alongside, same era.

Benzinga, “Microsoft defies the AI plateau, pushing boundaries with new scaling laws,” Benzinga , 2024. [Online]. Available: https://www.benzinga.com/24/11/42088290/microsoft-defies-the-ai-plateau-pushing-boundaries-with-new-scaling-laws

2024

Cited alongside, same era.

W. Central, “There’s no evidence scaling laws have begun to stop, former Google CEO claims AI systems will be 100 times more powerful,” Windows Central , 2024. [Online]. Available: https://www.windowscentral.com/software-apps/theres-no-evidence-scaling-laws-have-begun-to-stop-former-google-ceo-claims-ai-systems-will-be-100-times-more-powerful

2024

Cited alongside, same era.

N. Sardana, J. Portes, S. Doubov, and J. Frankle, “Beyond Chinchilla-optimal: Accounting for inference-time in language model scaling,” arXiv preprint arXiv:2211.02011 , 2024. [Online]. Available: https://arxiv.org/abs/2401.00448

Original

2024

Cited alongside, same era.

A. Grattafiori, A. Dubey, A. Jauhri, A. Pandey, A. Kadian, A. Al-Dahle, A. Letman, A. Mathur et al. , “The llama 3 herd of models,” arXiv preprint arXiv:2407.21783 . [Online]. Available: https://arxiv.org/abs/2407.21783

Original

Cited in the paper.

NVIDIA Blog, “Can innovation curb AI’s hunger for power?” https://blogs.nvidia.com/blog/ai-energy-efficiency/ , 2024, accessed: December 29, 2024

2024

Later among the works it cites.

Y. Wu, Z. Sun, S. Li, S. Welleck, and Y. Yang, “Inference scaling laws: An empirical analysis of compute-optimal inference for LLM problem-solving,” arXiv preprint arXiv:2408.00724 , 2024. [Online]. Available: https://arxiv.org/abs/2408.00724

Original

2024

Later among the works it cites.

E. Zelikman, Y. Wu, and N. D. Goodman, “Quiet-STaR: Language models can teach themselves to think before speaking,” arXiv preprint arXiv:2403.09629 , 2024. [Online]. Available: https://arxiv.org/abs/2403.09629

Original

2024

Later among the works it cites.

NVIDIA, NVIDIA Puts Grace Blackwell on Every Desk and at Every AI Developer’s Fingertips , 2025. [Online]. Available: https://nvidianews.nvidia.com/news/nvidia-puts-grace-blackwell-on-every-desk-and-at-every-ai-developers-fingertips

2025

Closest in time.