Fetching the paper…
Reading the bibliography…
Training deep neural networks (DNNs) is a major workload in datacenters today, resulting in a tremendously fast growth of energy consumption.
Pareto optimality in multiobjective problems
Yair Censor · 1977
Earlier work this paper cites.
Supply and threshold voltage scaling for low power CMOS
Ricardo Gonzalez, Benjamin M. Gordon, and Mark A. Horowitz · 1997
Earlier work this paper cites.
A static power model for architects
J Adam Butts and Gurindar S Sohi · 2000
Earlier work this paper cites.
Imagenet: A large-scale hierarchical image database
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Fei-Fei Li · 2009
Earlier work this paper cites.
Learning word vectors for sentiment analysis
Andrew L. Maas, Raymond E. Daly, Peter T. Pham, Dan Huang, Andrew Y. Ng, and Christopher Potts · 2011
Earlier work this paper cites.
Large scale distributed deep networks
Jeffrey Dean, Greg Corrado, Rajat Monga, Kai Chen, Matthieu Devin, Quoc V. Le, Mark Z. Mao, Marc’Aurelio Ranzato, Andrew W. Senior, Paul A. Tucker, Ke Yang, and Andrew Y. Ng · 2012
Earlier work this paper cites.
Imagenet classification with deep convolutional neural networks
Alex Krizhevsky, Ilya Sutskever, and Geoffrey E. Hinton · 2012
Earlier work this paper cites.
Power capping of CPU-GPU heterogeneous systems through coordinating DVFS and task mapping
Toshiya Komoda, Shingo Hayashi, Takashi Nakada, Shinobu Miwa, and Hiroshi Nakamura · 2013
Earlier work this paper cites.
Apache hadoop YARN: yet another resource negotiator
Vinod Kumar Vavilapalli, Arun C. Murthy, Chris Douglas, Sharad Agarwal, Mahadev Konar, Robert Evans, Thomas Graves, Jason Lowe, Hitesh Shah, Siddharth Seth, Bikas Saha, Carlo Curino, Owen O’Malley, Sanjay Radia, Benjamin Reed, and Eric Baldeschwieler · 2013
Earlier work this paper cites.
Scaling distributed machine learning with the parameter server
Mu Li, David G Andersen, Jun Woo Park, Alexander J Smola, Amr Ahmed, Vanja Josifovski, James Long, Eugene J Shekita, and Bor-Yiing Su · 2014
Earlier work this paper cites.
Communication efficient distributed machine learning with the parameter server
Mu Li, David G. Andersen, Alexander J. Smola, and Kai Yu · 2014
Earlier work this paper cites.
Deepdriving: Learning affordance for direct perception in autonomous driving
Chenyi Chen, Ari Seff, Alain L. Kornhauser, and Jianxiong Xiao · 2015
Earlier work this paper cites.
Energy efficient scheduling of virtual machines in cloud with deadline constraint
Youwei Ding, Xiaolin Qin, Liang Liu, and Taochun Wang · 2015
Earlier work this paper cites.
Librispeech: An ASR corpus based on public domain audio books
Vassil Panayotov, Guoguo Chen, Daniel Povey, and Sanjeev Khudanpur · 2015
Earlier work this paper cites.
Very deep convolutional networks for large-scale image recognition
Karen Simonyan and Andrew Zisserman · 2015
Earlier work this paper cites.
Deep speech 2 : End-to-end speech recognition in english and mandarin
Dario Amodei, Sundaram Ananthanarayanan, Rishita Anubhai, Jingliang Bai, Eric Battenberg, Carl Case, Jared Casper, Bryan Catanzaro, Jingdong Chen, Mike Chrzanowski, Adam Coates, Greg Diamos, Erich Elsen, Jesse H. Engel, Linxi Fan, Christopher Fougner, Awni Y. Hannun, Billy Jun, Tony Han, Patrick LeGresley, Xiangang Li, Libby Lin, Sharan Narang, Andrew Y. Ng, Sherjil Ozair, Ryan Prenger, Sheng Qian, Jonathan Raiman, Sanjeev Satheesh, David Seetapun, Shubho Sengupta, Chong Wang, Yi Wang, Zhiqian Wang, Bo Xiao, Yan Xie, Dani Yogatama, Jun Zhan, and Zhenyao Zhu · 2016
Earlier work this paper cites.
Listen and translate: A proof of concept for end-to-end speech-to-text translation
Alexandre Berard, Olivier Pietquin, Christophe Servan, and Laurent Besacier · 2016
Earlier work this paper cites.
Deep residual learning for image recognition
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun · 2016
Earlier work this paper cites.
Evaluating the energy efficiency of deep convolutional neural networks on cpus and gpus
Da Li, Xinbo Chen, Michela Becchi, and Ziliang Zong · 2016
Earlier work this paper cites.
Rethinking the inception architecture for computer vision
Christian Szegedy, Vincent Vanhoucke, Sergey Ioffe, Jonathon Shlens, and Zbigniew Wojna · 2016
Earlier work this paper cites.
Scalable multi-framework multi-tenant lifecycle management of deep learning training jobs
Scott Boag, Parijat Dube, Benjamin Herta, Waldemar Hummer, Vatche Ishakian, K Jayaram, Michael Kalantar, Vinod Muthusamy, Priya Nagpurkar, and Florian Rosenberg · 2017
Earlier work this paper cites.
A survey and measurement study of GPU DVFS on energy conservation
Xinxin Mei, Qiang Wang, and Xiaowen Chu · 2017
Earlier work this paper cites.
GPGPU power modeling for multi-domain voltage-frequency scaling
João Guerreiro, Aleksandar Ilic, Nuno Roma, and Pedro Tomás · 2018
Earlier work this paper cites.
Multi-tenant GPU clusters for deep learning workloads: Analysis and implications
Myeongjae Jeon, Shivaram Venkataraman, Junjie Qian, Amar Phanishayee, Wencong Xiao, and Fan Yang · 2018
Cited alongside, same era.
Optimus: an efficient dynamic resource scheduler for deep learning clusters
Yanghua Peng, Yixin Bao, Yangrui Chen, Chuan Wu, and Chuanxiong Guo · 2018
Cited alongside, same era.
A learning automata-based algorithm for energy and SLA efficient consolidation of virtual machines in cloud data centers
Milad Ranjbari and Javad Akbari Torkestani · 2018
Cited alongside, same era.
Energy efficient job scheduling with workload prediction on cloud data center
Xiaoyong Tang, Xiaoyi Liao, Jie Zheng, and Xiaopan Yang · 2018
Cited alongside, same era.
Robust optimization for household load scheduling with uncertain parameters
Jidong Wang, Peng Li, Kaijie Fang, and Yue Zhou · 2018
Cited alongside, same era.
Energy and policy considerations for modern deep learning research
Emma Strubell, Ananya Ganesh, and Andrew McCallum · 2020
Later among the works it cites.
Hived: Sharing a GPU cluster for deep learning with guarantees
Hanyu Zhao, Zhenhua Han, Zhi Yang, Quanlu Zhang, Fan Yang, Lidong Zhou, Mao Yang, Francis C. M. Lau, Yuqi Wang, Yifan Xiong, and Bin Wang · 2020
Later among the works it cites.
DUB: dynamic underclocking and bypassing in nocs for heterogeneous GPU workloads
Srikant Bharadwaj, Shomit Das, Yasuko Eckert, Mark Oskin, and Tushar Krishna · 2021
Later among the works it cites.
Elastic resource sharing for distributed deep learning
Changho Hwang, Taehyun Kim, Sunghyun Kim, Jinwoo Shin, and KyoungSoo Park · 2021
Later among the works it cites.
Elastic resource sharing for distributed deep learning
Changho Hwang, Taehyun Kim, Sunghyun Kim, Jinwoo Shin, and KyoungSoo Park · 2021
Later among the works it cites.
Zico: Efficient GPU memory sharing for concurrent DNN training
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Gandiva: Introspective cluster scheduling for deep learning
Wencong Xiao, Romil Bhardwaj, Ramachandran Ramjee, Muthian Sivathanu, Nipun Kwatra, Zhenhua Han, Pratyush Patel, Xuan Peng, Hanyu Zhao, Quanlu Zhang, Fan Yang, and Lidong Zhou · 2018
Cited alongside, same era.
BERT: pre-training of deep bidirectional transformers for language understanding
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova · 2019
Cited alongside, same era.
Tiresias: A GPU cluster manager for distributed deep learning
Juncheng Gu, Mosharaf Chowdhury, Kang G. Shin, Yibo Zhu, Myeongjae Jeon, Junjie Qian, Hongqiang Harry Liu, and Chuanxiong Guo · 2019
Cited alongside, same era.
Towards power efficiency in deep learning on data center hardware
Miro Hodak, Masha Gorkovenko, and Ajay Dholakia · 2019
Cited alongside, same era.
Gpipe: Efficient training of giant neural networks using pipeline parallelism
Yanping Huang, Youlong Cheng, Ankur Bapna, Orhan Firat, Dehao Chen, Mia Xu Chen, HyoukJoong Lee, Jiquan Ngiam, Quoc V. Le, Yonghui Wu, and Zhifeng Chen · 2019
Cited alongside, same era.
Analysis of large-scale multi-tenant GPU clusters for DNN training workloads
Myeongjae Jeon, Shivaram Venkataraman, Amar Phanishayee, Junjie Qian, Wencong Xiao, and Fan Yang · 2019
Cited alongside, same era.
Beyond data and model parallelism for deep neural networks
Zhihao Jia, Matei Zaharia, and Alex Aiken · 2019
Cited alongside, same era.
Gangmuk Lim, Jeongseob Ahn, Wencong Xiao, Youngjin Kwon, and Myeongjae Jeon · 2021
Later among the works it cites.
Carbon emissions and large neural network training
David A. Patterson, Joseph Gonzalez, Quoc V. Le, Chen Liang, Lluis-Miquel Munguia, Daniel Rothchild, David R. So, Maud Texier, and Jeff Dean · 2021
Later among the works it cites.
Pollux: Co-adaptive cluster scheduling for goodput-optimized deep learning
Aurick Qiao, Sang Keun Choe, Suhas Jayaram Subramanya, Willie Neiswanger, Qirong Ho, Hao Zhang, Gregory R. Ganger, and Eric P. Xing · 2021
Later among the works it cites.
Energy-efficient VM scheduling based on deep reinforcement learning
Bin Wang, Fagui Liu, and Weiwei Lin · 2021
Later among the works it cites.
https://openai.com/blog/ai-and-compute/ , 2019
AI and Compute · 2022
Later among the works it cites.
https://developer.nvidia.com/nvidia-management-library-nvml , 2019
NVIDIA Management Library (NVML) · 2022
Later among the works it cites.
https://www.technologyreview.com/2019/06/06/239031/training-a-single-ai-model-can-emit-as-much-carbon-as-five-cars-in-their-lifetimes/ , 2019
Training a single AI model can emit as much carbon as five cars in their lifetimes · 2022
Later among the works it cites.
https://www.nasdaq.com/articles/google-aims-to-attain-zero-carbon-footprint-goal-by-2030-2020-09-15 , 2020
Google Aims to Attain Zero Carbon Footprint Goal by 2030 · 2022
Later among the works it cites.
https://blogs.microsoft.com/blog/2020/01/16/microsoft-will-be-carbon-negative-by-2030/ , 2020
Microsoft will be carbon negative by 2030 · 2022
Later among the works it cites.
https://www.cbsnews.com/news/facebook-renewable-energy-commitment-100-percent-milestone/ , 2021
Facebook reaches 100% renewable-energy milestone · 2022
Later among the works it cites.
https://openai.com/blog/gpt-3-apps/ , 2021
GPT-3 Powers the Next Generation of Apps · 2022
Later among the works it cites.
https://www.intel.com/content/www/us/en/gaming/resources/turbo-boost.html , 2021
What Is Intel Turbo Boost Technology? · 2022
Later among the works it cites.
Measuring the carbon intensity of AI in cloud instances
Jesse Dodge, Taylor Prewitt, Remi Tachet des Combes, Erika Odmark, Roy Schwartz, Emma Strubell, Alexandra Sasha Luccioni, Noah A. Smith, Nicole DeCario, and Will Buchanan · 2022
Later among the works it cites.
Looking beyond gpus for DNN scheduling on multi-tenant clusters
Jayashree Mohan, Amar Phanishayee, Janardhan Kulkarni, and Vijay Chidambaram · 2022
Later among the works it cites.
Mlaas in the wild: Workload analysis and scheduling in large-scale heterogeneous GPU clusters
Qizhen Weng, Wencong Xiao, Yinghao Yu, Wei Wang, Cheng Wang, Jian He, Yong Li, Liping Zhang, Wei Lin, and Yu Ding · 2022
Later among the works it cites.
Zeus: Understanding and optimizing GPU energy consumption of DNN training
Jie You, Jae-Won Chung, and Mosharaf Chowdhury · 2022
Later among the works it cites.
Multi-resource interleaving for deep learning training
Yihao Zhao, Yuanqiang Liu, Yanghua Peng, Yibo Zhu, Xuanzhe Liu, and Xin Jin · 2022
Later among the works it cites.