Fetching the paper…

Reading the bibliography…

2020

Towards Accurate and Reliable Energy Measurement of NLP Models

Cao, Qingqing, Balasubramanian, Aruna, Balasubramanian, Niranjan

Original
PDF
Post on X

Understand

Accurate and reliable measurement of energy consumption is critical for making well-informed design choices when choosing and training large scale NLP models.

In this work, we show that existing software-based energy measurements are not accurate because they do not take into account hardware differences and how resource utilization affects energy consumption.
We conduct energy measurement experiments with four different models for a question answering task.
We quantify the error of existing software-based energy measurements by using a hardware power meter that provides highly accurate energy measurements.

Towards Accurate and Reliable Energy Measurement of NLP Models · Around

Built on

Tackling Climate Change with Machine Learning
Original
David Rolnick, Priya L. Donti, Lynn H. Kaack, Kelly Kochanski, Alexandre Lacoste, Kris Sankaran, Andrew Slavin Ross, Nikola Milojevic-Dupont, Natasha Jaques, Anna Waldman-Brown, Alexandra Luccioni, Tegan Maharaj, Evan D. Sherwin, S. Karthik Mukkavilli, Konrad P. Kording, Carla Gomes, Andrew Y. Ng, Demis Hassabis, John C. Platt, Felix Creutzig, Jennifer Chayes, and Yoshua Bengio. 2019 · 1906
Earlier work this paper cites.
RoBERTa: A Robustly Optimized BERT Pretraining Approach
Original
Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019 · 1907
Earlier work this paper cites.
Green AI
Original
Roy Schwartz, Jesse Dodge, Noah A. Smith, and Oren Etzioni. 2019 · 1907
Earlier work this paper cites.
DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter
Original
Victor Sanh, Lysandre Debut, Julien Chaumond, and Thomas Wolf. 2020 · 1910
Earlier work this paper cites.
HuggingFace’s Transformers: State-of-the-art Natural Language Processing
Original
Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mariama Drame, Quentin Lhoest, and Alexander M. Rush. 2020 · 1910
Earlier work this paper cites.
Towards the Systematic Reporting of the Energy and Carbon Footprints of Machine Learning
Original
Peter Henderson, Jieru Hu, Joshua Romoff, Emma Brunskill, Dan Jurafsky, and Joelle Pineau. 2020 · 2002
Earlier work this paper cites.
HULK: An Energy Efficiency Benchmark Platform for Responsible Natural Language Processing
Original
Xiyou Zhou, Zhiyu Chen, Xiaoyong Jin, and William Yang Wang. 2020 · 2002
Earlier work this paper cites.

Similar

MobileBERT: a Compact Task-Agnostic BERT for Resource-Limited Devices
Original
Zhiqing Sun, Hongkun Yu, Xiaodan Song, Renjie Liu, Yiming Yang, and Denny Zhou. 2020 · 2004
Cited alongside, same era.
Fine-grained power modeling for smartphones using system call tracing
Abhinav Pathak, Y. Charlie Hu, Ming Zhang, Paramvir Bahl, and Yi-Min Wang. 2011 · 2011
Cited alongside, same era.
Where is the energy spent inside my app? fine grained energy accounting on smartphones with Eprof
Abhinav Pathak, Y. Charlie Hu, and Ming Zhang. 2012 · 2012
Cited alongside, same era.
AppScope: application energy metering framework for android smartphones using kernel activity monitoring
Chanmin Yoon, Dongwon Kim, Wonwoo Jung, Chulkoo Kang, and Hojung Cha. 2012 · 2012
Cited alongside, same era.
Measuring GPU Power with the K20 Built-in Sensor
Martin Burtscher, Ivan Zecena, and Ziliang Zong. 2014 · 2014
Cited alongside, same era.

Then

Power-performance tradeoffs in data center servers: DVFS, CPU pinning, horizontal, and vertical scaling
Jakub Krzywda, Ahmed Ali-Eldin, Trevor E. Carlson, Per-Olov Östberg, and Erik Elmroth. 2018 · 2018
Later among the works it cites.
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019 · 2019
Later among the works it cites.
Estimation of energy consumption in machine learning
Eva García-Martín, Crefeda Faviola Rodrigues, Graham Riley, and Håkan Grahn. 2019 · 2019
Later among the works it cites.
PyTorch: An Imperative Style, High-Performance Deep Learning Library
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Kopf, Edward Yang, Zachary DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. 2019 · 2019
Later among the works it cites.
Energy and Policy Considerations for Deep Learning in NLP
Emma Strubell, Ananya Ganesh, and Andrew McCallum. 2019 · 2019
Later among the works it cites.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

SQuAD: 100,000+ Questions for Machine Comprehension of Text

Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, and Percy Liang. 2016 · 2016

Cited alongside, same era.

Deconstructing the Energy Consumption of the Mobile Page Load

Yi Cao, Javad Nejati, Muhammad Wajahat, Aruna Balasubramanian, and Anshul Gandhi. 2017 · 2017

Cited alongside, same era.

Global Warming of 1.5 ºC

Cited in the paper.

WattsUp Meter Pro

Cited in the paper.

Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer

Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. 2020 · 2020

Closest in time.