The Unseen AI Disruptions for Power Grids: LLM-Induced Transients

B. Kirby and M. Milligan, “Method and case study for estimating the ramping capability of a control area or balancing authority and implications for moderate or high wind penetration,” National Renewable Energy Lab., Golden, CO (US), Tech. Rep., 2005

2005

Earlier work this paper cites.

D. Salomonsson, L. Soder, and A. Sannino, “An adaptive control system for a dc microgrid for data centers,” in 2007 IEEE Industry Applications Annual Meeting . IEEE, 2007, pp. 2414–2421

2007

Earlier work this paper cites.

P. Wang, P. McCluskey, and A. Bar-Cohen, “Two-phase liquid cooling for thermal management of igbt power electronic module,” Journal of Electronic Packaging , vol. 135, no. 2, p. 021001, 2013

2013

Earlier work this paper cites.

F. Kong and X. Liu, “A survey on green-energy-aware power management for datacenters,” ACM Computing Surveys (CSUR) , vol. 47, no. 2, pp. 1–38, 2014

2014

Earlier work this paper cites.

E. Oró, V. Depoorter, A. Garcia, and J. Salom, “Energy efficiency and renewable energy integration in data centres. strategies and modelling review,” Renewable and Sustainable Energy Reviews , vol. 42, pp. 429–445, 2015. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S1364032114008600

2015

Earlier work this paper cites.

J. Zhang, F. R. Yu, S. Wang, T. Huang, Z. Liu, and Y. Liu, “Load balancing in data center networks: A survey,” IEEE Communications Surveys & Tutorials , vol. 20, no. 3, pp. 2324–2352, 2018

2018

Earlier work this paper cites.

J. Devlin, “Bert: Pre-training of deep bidirectional transformers for language understanding,” arXiv preprint arXiv:1810.04805 , 2018

Original

2018

Earlier work this paper cites.

A. Lacoste, A. Luccioni, V. Schmidt, and T. Dandres, “Quantifying the carbon emissions of machine learning,” arXiv preprint arXiv:1910.09700 , 2019

Original

2019

Earlier work this paper cites.

Z. Jia, M. Zaharia, and A. Aiken, “Beyond data and model parallelism for deep neural networks.” Proceedings of Machine Learning and Systems , vol. 1, pp. 1–13, 2019

2019

Earlier work this paper cites.

S. Kwon, “Ensuring renewable energy utilization with quality of service guarantee for energy-efficient data center operations,” Applied Energy , vol. 276, p. 115424, 2020. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S0306261920309363

2020

Earlier work this paper cites.

L. F. W. Anthony, B. Kanding, and R. Selvan, “Carbontracker: Tracking and predicting the carbon footprint of training deep learning models,” arXiv preprint arXiv:2007.03051 , 2020

Original

2020

Earlier work this paper cites.

P. Henderson, J. Hu, J. Romoff, E. Brunskill, D. Jurafsky, and J. Pineau, “Towards the systematic reporting of the energy and carbon footprints of machine learning,” Journal of Machine Learning Research , vol. 21, no. 248, pp. 1–43, 2020

2020

Earlier work this paper cites.

S. McQueen, J. Stanford, S. Satyapal, E. Miller, N. Stetson, D. Papageorgopoulos, N. Rustagi, V. Arjona, J. Adams, K. Randolph et al. , “Department of energy hydrogen program plan,” US Department of Energy (USDOE), Washington DC (United States), Tech. Rep., 2020

2020

Earlier work this paper cites.

R. Bommasani, D. A. Hudson, E. Adeli, R. Altman, S. Arora, S. von Arx, M. S. Bernstein, J. Bohg, A. Bosselut, E. Brunskill et al. , “On the opportunities and risks of foundation models,” arXiv preprint arXiv:2108.07258 , 2021

Original

2021

Earlier work this paper cites.

M. Lui, Y. Yetim, Ö. Özkan, Z. Zhao, S.-Y. Tsai, C.-J. Wu, and M. Hempstead, “Understanding capacity-driven scale-out neural recommendation inference,” in 2021 IEEE International Symposium on Performance Analysis of Systems and Software (ISPASS) . IEEE, 2021, pp. 162–171

2021

Earlier work this paper cites.

A. Radovanovic, B. Chen, S. Talukdar, B. Roy, A. Duarte, and M. Shahbazi, “Power modeling for effective datacenter planning and compute management,” IEEE Transactions on Smart Grid , vol. 13, no. 2, pp. 1611–1621, 2021

2021

Earlier work this paper cites.

S. Samsi, M. L. Weiss, D. Bestor, B. Li, M. Jones, A. Reuther, D. Edelman, W. Arcand, C. Byun, J. Holodnack et al. , “The mit supercloud dataset,” in 2021 IEEE High Performance Extreme Computing Conference (HPEC) . IEEE, 2021, pp. 1–8

2021

Earlier work this paper cites.

L. H. Kaack, P. L. Donti, E. Strubell, G. Kamiya, F. Creutzig, and D. Rolnick, “Aligning artificial intelligence with climate change mitigation,” Nature Climate Change , vol. 12, no. 6, pp. 518–527, 2022

2022

Earlier work this paper cites.

J. Sevilla, L. Heim, A. Ho, T. Besiroglu, M. Hobbhahn, and P. Villalobos, “Compute trends across three eras of machine learning,” in 2022 International Joint Conference on Neural Networks (IJCNN) . IEEE, 2022, pp. 1–8

2022

Earlier work this paper cites.

B. Li, R. Arora, S. Samsi, T. Patel, W. Arcand, D. Bestor, C. Byun, R. B. Roy, B. Bergeron, J. Holodnak et al. , “Ai-enabling workloads on large-scale gpu-accelerated system: Characterization, opportunities, and implications,” in 2022 IEEE International Symposium on High-Performance Computer Architecture (HPCA) . IEEE, 2022, pp. 1224–1237

2022

Earlier work this paper cites.

J. McDonald, B. Li, N. Frey, D. Tiwari, V. Gadepally, and S. Samsi, “Great power, great responsibility: Recommendations for reducing energy for training language models,” arXiv preprint arXiv:2205.09646 , 2022

Original

2022

Cited alongside, same era.

C.-J. Wu, R. Raghavendra, U. Gupta, B. Acun, N. Ardalani, K. Maeng, G. Chang, F. Aga, J. Huang, C. Bai et al. , “Sustainable ai: Environmental implications, challenges and opportunities,” Proceedings of Machine Learning and Systems , vol. 4, pp. 795–813, 2022

2022

Cited alongside, same era.

Y. R. Li, F. Nejabatkhah, and H. Tian, Smart hybrid AC/DC microgrids: power management, energy management, and power quality control . John Wiley & Sons, 2022

2022

Cited alongside, same era.

S. Bubeck, V. Chandrasekaran, R. Eldan, J. Gehrke, E. Horvitz, E. Kamar, P. Lee, Y. T. Lee, Y. Li, S. Lundberg et al. , “Sparks of artificial general intelligence: Early experiments with gpt-4,” arXiv preprint arXiv:2303.12712 , 2023

Original

2023

Cited alongside, same era.

ML Energy, “Ml energy leaderboard,” https://ml.energy/leaderboard/?__theme=light , 2024, accessed: 2024-09-07

2024

Closest in time.

A. Llama Team, “The llama 3 herd of models,” July 2024, a detailed contributor list can be found in the appendix of this paper. [Online]. Available: https://llama.meta.com/

2024

Closest in time.

A. Elmeleegy, S. Raj, B. Slechta, and V. Mehta, “Demystifying ai inference deployments for trillion parameter large language models,” https://developer.nvidia.com/blog/demystifying-ai-inference-deployments-for-trillion-parameter-large-language-models/ , June 2024, technical Blog, Data Center / Cloud, NVIDIA. Accessed: 2024-09-07

2024

Closest in time.

P. Patel, E. Choukse, C. Zhang, Í. Goiri, B. Warrier, N. Mahalingam, and R. Bianchini, “Characterizing power management opportunities for llms in the cloud,” in Proceedings of the 29th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Volume 3 , 2024, pp. 207–222