Toward Sustainable GenAI using Generation Directives for Carbon-Friendly Large Language Model Inference

J. Chang, J. Meza, P. Ranganathan, A. Shah, R. Shih, and C. Bash, “Totally green: evaluating and designing servers for lifecycle environmental impact,” ACM SIGPLAN Notices , vol. 47, no. 4, pp. 25–36, 2012

2012

Earlier work this paper cites.

J. Charan and T. Biswas, “How to calculate sample size for different study designs in medical research?” Indian journal of psychological medicine , vol. 35, no. 2, pp. 121–126, 2013

2013

Earlier work this paper cites.

M. Joshi, E. Choi, D. S. Weld, and L. Zettlemoyer, “Triviaqa: A large scale distantly supervised challenge dataset for reading comprehension,” arXiv preprint arXiv:1705.03551 , 2017

Original

2017

Earlier work this paper cites.

D. Crankshaw, X. Wang, G. Zhou, M. J. Franklin, J. E. Gonzalez, and I. Stoica, “Clipper: A { \{ Low-Latency } \} online prediction serving system,” in 14th USENIX Symposium on Networked Systems Design and Implementation (NSDI 17) , 2017, pp. 613–627

2017

Earlier work this paper cites.

J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, “Bert: Pre-training of deep bidirectional transformers for language understanding,” arXiv preprint arXiv:1810.04805 , 2018

Original

2018

Earlier work this paper cites.

Q. Huangfu and J. J. Hall, “Parallelizing the dual revised simplex method,” Mathematical Programming Computation , vol. 10, no. 1, pp. 119–142, 2018

2018

Earlier work this paper cites.

T. Kwiatkowski, J. Palomaki, O. Redfield, M. Collins, A. Parikh, C. Alberti, D. Epstein, I. Polosukhin, J. Devlin, K. Lee et al. , “Natural questions: a benchmark for question answering research,” Transactions of the Association for Computational Linguistics , vol. 7, pp. 453–466, 2019

2019

Earlier work this paper cites.

E. Strubell, A. Ganesh, and A. McCallum, “Energy and policy considerations for deep learning in nlp,” in Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics , 2019, pp. 3645–3650

2019

Earlier work this paper cites.

C. Wan, M. Santriaji, E. Rogers, H. Hoffmann, M. Maire, and S. Lu, “ { \{ ALERT } \} : Accurate learning for energy and timeliness,” in 2020 USENIX annual technical conference (USENIX ATC 20) , 2020, pp. 353–369

2020

Earlier work this paper cites.

D. Hendrycks, C. Burns, S. Basart, A. Zou, M. Mazeika, D. Song, and J. Steinhardt, “Measuring massive multitask language understanding,” arXiv preprint arXiv:2009.03300 , 2020

Original

2020

Earlier work this paper cites.

L. F. W. Anthony, B. Kanding, and R. Selvan, “Carbontracker: Tracking and predicting the carbon footprint of training deep learning models,” arXiv preprint arXiv:2007.03051 , 2020

Original

2020

Earlier work this paper cites.

R. Schwartz, J. Dodge, N. A. Smith, and O. Etzioni, “Green ai,” Communications of the ACM , vol. 63, no. 12, pp. 54–63, 2020

2020

Earlier work this paper cites.

J. Jumper, R. Evans, A. Pritzel, T. Green, M. Figurnov, O. Ronneberger, K. Tunyasuvunakool, R. Bates, A. Žídek, A. Potapenko et al. , “Highly accurate protein structure prediction with alphafold,” Nature , vol. 596, no. 7873, pp. 583–589, 2021

2021

Earlier work this paper cites.

F. Romero, Q. Li, N. J. Yadwadkar, and C. Kozyrakis, “ { \{ INFaaS } \} : Automated model-less inference serving,” in 2021 USENIX Annual Technical Conference (USENIX ATC 21) , 2021, pp. 397–411

2021

Earlier work this paper cites.

U. Gupta, Y. G. Kim, S. Lee, J. Tse, H.-H. S. Lee, G.-Y. Wei, D. Brooks, and C.-J. Wu, “Chasing carbon: The elusive environmental footprint of computing,” in 2021 IEEE International Symposium on High-Performance Computer Architecture (HPCA) . IEEE, 2021, pp. 854–867

2021

Earlier work this paper cites.

K. Cobbe, V. Kosaraju, M. Bavarian, M. Chen, H. Jun, L. Kaiser, M. Plappert, J. Tworek, J. Hilton, R. Nakano et al. , “Training verifiers to solve math word problems,” arXiv preprint arXiv:2110.14168 , 2021

Original

2021

Earlier work this paper cites.

D. Patterson, J. Gonzalez, Q. Le, C. Liang, L.-M. Munguia, D. Rothchild, D. So, M. Texier, and J. Dean, “Carbon emissions and large neural network training,” arXiv preprint arXiv:2104.10350 , 2021

Original

2021

Earlier work this paper cites.

S. Rajbhandari, O. Ruwase, J. Rasley, S. Smith, and Y. He, “Zero-infinity: Breaking the gpu memory wall for extreme scale deep learning,” in Proceedings of the international conference for high performance computing, networking, storage and analysis , 2021, pp. 1–14

2021

Earlier work this paper cites.

U. Gupta, M. Elgamal, G. Hills, G.-Y. Wei, H.-H. S. Lee, D. Brooks, and C.-J. Wu, “Act: Designing sustainable computer systems with an architectural carbon modeling tool,” in Proceedings of the 49th Annual International Symposium on Computer Architecture , 2022, pp. 784–799

2022

Earlier work this paper cites.

C.-J. Wu, R. Raghavendra, U. Gupta, B. Acun, N. Ardalani, K. Maeng, G. Chang, F. Aga, J. Huang, C. Bai et al. , “Sustainable ai: Environmental implications, challenges and opportunities,” Proceedings of Machine Learning and Systems , vol. 4, pp. 795–813, 2022

2022

Earlier work this paper cites.

J. Dodge, T. Prewitt, R. Tachet des Combes, E. Odmark, R. Schwartz, E. Strubell, A. S. Luccioni, N. A. Smith, N. DeCario, and W. Buchanan, “Measuring the carbon intensity of ai in cloud instances,” in Proceedings of the 2022 ACM conference on fairness, accountability, and transparency , 2022, pp. 1877–1894

2022

Earlier work this paper cites.

G.-I. Yu, J. S. Jeong, G.-W. Kim, S. Kim, and B.-G. Chun, “Orca: A distributed serving system for { \{ Transformer-Based } \} generative models,” in 16th USENIX Symposium on Operating Systems Design and Implementation (OSDI 22) , 2022, pp. 521–538

2022

Earlier work this paper cites.

P. Lu, S. Mishra, T. Xia, L. Qiu, K.-W. Chang, S.-C. Zhu, O. Tafjord, P. Clark, and A. Kalyan, “Learn to explain: Multimodal reasoning via thought chains for science question answering,” Advances in Neural Information Processing Systems , vol. 35, pp. 2507–2521, 2022

2022

Earlier work this paper cites.

M. Dietrich and A. Chien, “Navigating dennard, carbon and moore: Scenarios for the future of nsf advanced computational infrastructure,” in Practice and Experience in Advanced Research Computing , 2022, pp. 1–6

2022

Earlier work this paper cites.

D. Patterson, J. Gonzalez, U. Hölzle, Q. Le, C. Liang, L.-M. Munguia, D. Rothchild, D. R. So, M. Texier, and J. Dean, “The carbon footprint of machine learning training will plateau, then shrink,” Computer , vol. 55, no. 7, pp. 18–28, 2022

2022

Earlier work this paper cites.

S. Choi, S. Lee, Y. Kim, J. Park, Y. Kwon, and J. Huh, “Serving heterogeneous machine learning models on { \{ Multi-GPU } \} servers with { \{ Spatio-Temporal } \} sharing,” in 2022 USENIX Annual Technical Conference (USENIX ATC 22) , 2022, pp. 199–216

2022

Earlier work this paper cites.

R. Y. Aminabadi, S. Rajbhandari, A. A. Awan, C. Li, D. Li, E. Zheng, O. Ruwase, S. Smith, M. Zhang, J. Rasley et al. , “Deepspeed-inference: enabling efficient inference of transformer models at unprecedented scale,” in SC22: International Conference for High Performance Computing, Networking, Storage and Analysis . IEEE, 2022, pp. 1–15

2022

Cited alongside, same era.

Z. Zhou, X. Wei, J. Zhang, and G. Sun, “ { \{ PetS } \} : A unified framework for { \{ Parameter-Efficient } \} transformers serving,” in 2022 USENIX Annual Technical Conference (USENIX ATC 22) , 2022, pp. 489–504

2022

Cited alongside, same era.

N. Pierce and S. Goutos, “Why law firms must responsibly embrace generative ai,” Available at SSRN 4477704 , 2023

2023

Cited alongside, same era.

B. Chen, Z. Wu, and R. Zhao, “From fiction to fact: the growing role of generative ai in business and finance,” Journal of Chinese Economic and Business Studies , vol. 21, no. 4, pp. 471–496, 2023

2023

Cited alongside, same era.

Z. Liu, J. Wang, T. Dao, T. Zhou, B. Yuan, Z. Song, A. Shrivastava, C. Zhang, Y. Tian, C. Re et al. , “Deja vu: Contextual sparsity for efficient llms at inference time,” in International Conference on Machine Learning . PMLR, 2023, pp. 22 137–22 176

2023

Later among the works it cites.

E. Frantar and D. Alistarh, “Sparsegpt: Massive language models can be accurately pruned in one-shot,” in International Conference on Machine Learning . PMLR, 2023, pp. 10 323–10 337

2023

Later among the works it cites.

Y. Leviathan, M. Kalman, and Y. Matias, “Fast inference from transformers via speculative decoding,” in International Conference on Machine Learning . PMLR, 2023, pp. 19 274–19 286

2023

Later among the works it cites.

C. Chen, S. Borgeaud, G. Irving, J.-B. Lespiau, L. Sifre, and J. Jumper, “Accelerating large language model decoding with speculative sampling,” arXiv preprint arXiv:2302.01318 , 2023

Original

2023

Later among the works it cites.