Fetching the paper…
Reading the bibliography…
A "bigger is better" explosion in the number of parameters in deep neural networks has made it increasingly challenging to make state-of-the-art networks accessible in compute-restricted environments.
The state of sparsity in deep neural networks
Trevor Gale, Erich Elsen, and Sara Hooker. 2019 · 1902
Earlier work this paper cites.
The State of Sparsity in Deep Neural Networks
Trevor Gale, Erich Elsen, and Sara Hooker. 2019 · 1902
Earlier work this paper cites.
A focus on neural machine translation for african languages
Laura Martinus and Jade Z. Abbott. 2019 · 1906
Earlier work this paper cites.
Massively multilingual neural machine translation in the wild: Findings and challenges
Naveen Arivazhagan, Ankur Bapna, Orhan Firat, Dmitry Lepikhin, Melvin Johnson, Maxim Krikun, Mia Xu Chen, Yuan Cao, George F. Foster, Colin Cherry, Wolfgang Macherey, Zhifeng Chen, and Yonghui Wu. 2019 · 1907
Earlier work this paper cites.
Resource-efficient machine learning in 2 KB RAM for the internet of things
Ashish Kumar, Saurabh Goyal, and Manik Varma. 2017 · 1944
Earlier work this paper cites.
Optimal brain damage
Yann Le Cun, John S. Denker, and Sara A. Solla. 1990 · 1990
Earlier work this paper cites.
Pruning algorithms-a survey
R. Reed. 1993 · 1993
Earlier work this paper cites.
Sparse connection and pruning in large dynamic artificial neural networks
Nikko Ström. 1997 · 1997
Earlier work this paper cites.
The Psycho-Biology of Language: An Introduction to Dynamic Philology
G.K. Zipf. 1999 · 1999
Earlier work this paper cites.
Bleu: A method for automatic evaluation of machine translation
Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002 · 2002
Earlier work this paper cites.
What is the state of neural network pruning?
Davis Blalock, Jose Javier Gonzalez Ortiz, Jonathan Frankle, and John Guttag. 2020 · 2003
Earlier work this paper cites.
Dynabert: Dynamic bert with adaptive width and depth
Lu Hou, Lifeng Shang, X. Jiang, and Qun Liu. 2020 · 2004
Earlier work this paper cites.
TICO-19: the translation initiative for covid-19
Antonios Anastasopoulos, Alessandro Cattelan, Zi-Yi Dou, Marcello Federico, Christian Federmann, Dmitriy Genzel, Francisco Guzmán, Junjie Hu, Macduff Hughes, Philipp Koehn, Rosie Lazar, William Lewis, Graham Neubig, Mengmeng Niu, Alp Öktem, Eric Paquin, Grace Tang, and Sylwia Tur. 2020 · 2007
Earlier work this paper cites.
The Computational Limits of Deep Learning
Neil C. Thompson, Kristjan Greenewald, Keeheon Lee, and Gabriel F. Manso. 2020 · 2007
Earlier work this paper cites.
Translationese and its dialects
Moshe Koppel and Noam Ordan. 2011 · 2011
Earlier work this paper cites.
Binarybert: Pushing the limit of bert quantization
Haoli Bai, Wei Zhang, Lu Hou, Lifeng Shang, Jing Jin, X. Jiang, Qun Liu, Michael R. Lyu, and Irwin King. 2020 · 2012
Earlier work this paper cites.
Learning light-weight translation models from deep transformer
Bei Li, Ziyang Wang, H. Liu, Quan Du, Tong Xiao, Chunliang Zhang, and Jingbo Zhu. 2020a · 2012
Earlier work this paper cites.
Memory Bounded Deep Convolutional Networks
Maxwell D. Collins and Pushmeet Kohli. 2014 · 2014
Earlier work this paper cites.
Training deep neural networks with low precision multiplications
Matthieu Courbariaux, Yoshua Bengio, and Jean-Pierre David. 2014 · 2014
Earlier work this paper cites.
1.1 computing’s energy problem (and what we can do about it)
M. Horowitz. 2014 · 2014
Earlier work this paper cites.
Deep learning with limited numerical precision
Suyog Gupta, Ankur Agrawal, Kailash Gopalakrishnan, and Pritish Narayanan. 2015 · 2015
Earlier work this paper cites.
Learning both Weights and Connections for Efficient Neural Network
Song Han, Jeff Pool, John Tran, and William J. Dally. 2015 · 2015
Earlier work this paper cites.
Distilling the Knowledge in a Neural Network
Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. 2015 · 2015
Earlier work this paper cites.
Selection criteria for low resource language programs
Christopher Cieri, Mike Maxwell, Stephanie Strassel, and Jennifer Tracey. 2016 · 2016
Earlier work this paper cites.
Dynamic Network Surgery for Efficient DNNs
Yiwen Guo, Anbang Yao, and Yurong Chen. 2016 · 2016
Earlier work this paper cites.
Quantized neural networks: Training neural networks with low precision weights and activations
Itay Hubara, Matthieu Courbariaux, Daniel Soudry, Ran El-Yaniv, and Yoshua Bengio. 2016 · 2016
Earlier work this paper cites.
SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and < < 0.5MB model size
F. N. Iandola, S. Han, M. W. Moskewicz, K. Ashraf, W. J. Dally, and K. Keutzer. 2016 · 2016
Earlier work this paper cites.
Compression of Neural Machine Translation Models via Pruning
Abigail See, Minh-Thang Luong, and Christopher D. Manning. 2016 · 2016
Earlier work this paper cites.
Compression of neural machine translation models via pruning
Abigail See, Minh-Thang Luong, and Christopher D. Manning. 2016 · 2016
Earlier work this paper cites.
Neural machine translation of rare words with subword units
Rico Sennrich, Barry Haddow, and Alexandra Birch. 2016 · 2016
Earlier work this paper cites.
Learning Structured Sparsity in Deep Neural Networks
W. Wen, C. Wu, Y. Wang, Y. Chen, and H. Li. 2016 · 2016
Cited alongside, same era.
Transfer learning for low-resource neural machine translation
Barret Zoph, Deniz Yuret, Jonathan May, and Kevin Knight. 2016 · 2016
Cited alongside, same era.
Dermatologist-level classification of skin cancer with deep neural networks
Andre Esteva, Brett Kuprel, Roberto Novoa, Justin Ko, Susan M Swetter, Helen M Blau, and Sebastian Thrun. 2017 · 2017
Cited alongside, same era.
Data augmentation for low-resource neural machine translation
Marzieh Fadaee, Arianna Bisazza, and Christof Monz. 2017 · 2017
Cited alongside, same era.
MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
A. G. Howard, M. Zhu, B. Chen, D. Kalenichenko, W. Wang, T. Weyand, M. Andreetto, and H. Adam. 2017 · 2017
Cited alongside, same era.
ParaCrawl: Web-scale acquisition of parallel corpora
Marta Bañón, Pinzhen Chen, Barry Haddow, Kenneth Heafield, Hieu Hoang, Miquel Esplà-Gomis, Mikel L. Forcada, Amir Kamran, Faheem Kirefu, Philipp Koehn, Sergio Ortiz Rojas, Leopoldo Pla Sempere, Gema Ramírez-Sánchez, Elsa Sarrías, Marek Strelec, Brian Thompson, William Waites, Dion Wiggins, and Jaume Zaragoza. 2020 · 2020
Later among the works it cites.
On optimal transformer depth for low-resource language translation
Elan Van Biljon, Arnu Pretorius, and Julia Kreutzer. 2020 · 2020
Later among the works it cites.
When is memorization of irrelevant training data necessary for high-accuracy learning?
Gavin Brown, Mark Bun, Vitaly Feldman, Adam Smith, and Kunal Talwar. 2020 · 2020
Later among the works it cites.
With little power comes great responsibility
Dallas Card, Peter Henderson, Urvashi Khandelwal, Robin Jia, Kyle Mahowald, and Dan Jurafsky. 2020 · 2020
Later among the works it cites.
Extremely low bit transformer quantization for on-device neural machine translation
Insoo Chung, Byeongwook Kim, Yoonjung Choi, Se Jung Kwon, Yongkweon Jeon, Baeseong Park, Sangha Kim, and Dongsoo Lee. 2020 · 2020
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Learning Sparse Neural Networks through L _ 0 L\_0 Regularization
C. Louizos, M. Welling, and D. P. Kingma. 2017 · 2017
Cited alongside, same era.
Exploring Sparsity in Recurrent Neural Networks
Sharan Narang, Erich Elsen, Gregory Diamos, and Shubho Sengupta. 2017 · 2017
Cited alongside, same era.
Exploring sparsity in recurrent neural networks
Sharan Narang, Erich Elsen, Gregory Diamos, and Shubho Sengupta. 2017 · 2017
Cited alongside, same era.
Attention is all you need
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017 · 2017
Cited alongside, same era.
Towards compact and fast neural machine translation using a combined method
Xiaowei Zhang, Wei Chen, Feng Wang, Shuang Xu, and Bo Xu. 2017 · 2017
Cited alongside, same era.
To prune, or not to prune: exploring the efficacy of pruning for model compression
Michael Zhu and Suyog Gupta. 2017 · 2017
Cited alongside, same era.
Ai and compute
Dario Amodei, Danny Hernandez, Girish Sastry, Jack Clark, Greg Brockman, and Ilya Sutskever. 2018 · 2018
Cited alongside, same era.
Later among the works it cites.
Benchmarking neural and statistical machine translation on low-resource African languages
Kevin Duh, Paul McNamee, Matt Post, and Brian Thompson. 2020 · 2020
Later among the works it cites.
On the evaluation of machine translation systems trained with back-translation
Sergey Edunov, Myle Ott, Marc’Aurelio Ranzato, and Michael Auli. 2020 · 2020
Later among the works it cites.
What neural networks memorize and why: Discovering the long tail via influence estimation
Vitaly Feldman and Chiyuan Zhang. 2020 · 2020
Later among the works it cites.
Sparse gpu kernels for deep learning
Trevor Gale, Matei Zaharia, Cliff Young, and Erich Elsen. 2020 · 2020
Later among the works it cites.
The state and fate of linguistic diversity and inclusion in the NLP world
Pratik Joshi, Sebastin Santy, Amar Budhiraja, Kalika Bali, and Monojit Choudhury. 2020 · 2020
Later among the works it cites.
Participatory research for low-resourced machine translation: A case study in African languages
Wilhelmina Nekoto, Vukosi Marivate, Tshinondiwa Matsila, Timi Fasubaa, Taiwo Fagbohungbe, Solomon Oluwole Akinola, Shamsuddeen Muhammad, Salomon Kabongo Kabenamualu, Salomey Osei, Freshia Sackey, Rubungo Andre Niyongabo, Ricky Macharm, Perez Ogayo, Orevaoghene Ahia, Musie Meressa Berhe, Mofetoluwa Adeyemi, Masabata Mokgesi-Selinga, Lawrence Okegbemi, Laura Martinus, Kolawole Tajudeen, Kevin Degila, Kelechi Ogueji, Kathleen Siminyu, Julia Kreutzer, Jason Webster, Jamiil Toure Ali, Jade Abbott, Iroro Orife, Ignatius Ezeani, Idris Abdulkadir Dangana, Herman Kamper, Hady Elsahar, Goodness Duru, Ghollah Kioko, Murhabazi Espoir, Elan van Biljon, Daniel Whitenack, Christopher Onyefuluchi, Chris Chinenye Emezue, Bonaventure F. P. Dossou, Blessing Sibanda, Blessing Bassey, Ayodele Olabiyi, Arshath Ramkilowan, Alp Öktem, Adewale Akinfaderin, and Abdallah Bashir. 2020 · 2020
Later among the works it cites.
Tigrinya neural machine translation with transfer learning for humanitarian response
Alp Öktem, Mirko Plitt, and Grace Tang. 2020 · 2020
Later among the works it cites.
On long-tailed phenomena in neural machine translation
Vikas Raunak, Siddharth Dalmia, Vivek Gupta, and Florian Metze. 2020 · 2020
Later among the works it cites.
Movement pruning: Adaptive sparsity by fine-tuning
Victor Sanh, Thomas Wolf, and Alexander M. Rush. 2020 · 2020
Later among the works it cites.
Computation on sparse neural networks and its implications for future hardware: Invited
Fei Sun, Minghai Qin, Tianyun Zhang, Liu Liu, Yen-Kuang Chen, and Yuan Xie. 2020 · 2020
Later among the works it cites.
Neural machine translation for extremely low-resource African languages: A case study on Bambara
Allahsera Auguste Tapo, Bakary Coulibaly, Sébastien Diarra, Christopher Homan, Julia Kreutzer, Sarah Luger, Arthur Nagashima, Marcos Zampieri, and Michael Leventhal. 2020 · 2020
Later among the works it cites.
Dynamic curriculum learning for low-resource neural machine translation
Chen Xu, Bojie Hu, Yufan Jiang, Kai Feng, Zeyang Wang, Shen Huang, Qi Ju, Tong Xiao, and Jingbo Zhu. 2020 · 2020
Later among the works it cites.
Keep the gradients flowing: Using gradient flow to study sparse network optimization
Kale ab Tessera, Sara Hooker, and Benjamin Rosman. 2021 · 2021
Closest in time.
On the dangers of stochastic parrots: Can language models be too big?
Emily M. Bender, Timnit Gebru, Angelina McMillan-Major, and Shmargaret Shmitchell. 2021 · 2021
Closest in time.
Earlybert: Efficient bert training via early-bird lottery tickets
Xiao-Han Chen, Yu Cheng, Shuohang Wang, Zhe Gan, Zhangyang Wang, and Jing jing Liu. 2021 · 2021
Closest in time.
Okwugbé: End-to-end speech recognition for fon and igbo
Bonaventure F. P. Dossou and Chris C. Emezue. 2021 · 2021
Closest in time.
The flores-101 evaluation benchmark for low-resource and multilingual machine translation
Naman Goyal, Cynthia Gao, Vishrav Chaudhary, Peng-Jen Chen, Guillaume Wenzek, Da Ju, Sanjana Krishnan, Marc’Aurelio Ranzato, Francisco Guzmán, and Angela Fan. 2021 · 2021
Closest in time.
Pitfalls of static language modelling
Angeliki Lazaridou, Adhiguna Kuncoro, Elena Gribovskaya, Devang Agrawal, Adam Liska, Tayfun Terzi, Mai Gimenez, Cyprien de Masson d’Autume, Sebastian Ruder, Dani Yogatama, Kris Cao, Tomas Kocisky, Susannah Young, and Phil Blunsom. 2021 · 2021
Closest in time.
Lost in pruning: The effects of pruning neural networks beyond test accuracy
Lucas Liebenwein, Cenk Baykal, Brandon Carter, David Gifford, and Daniela Rus. 2021 · 2021
Closest in time.
Congolese swahili machine translation for humanitarian response
Alp Öktem, Eric DeLuca, Rodrigue Bashizi, Eric Paquin, and Grace Tang. 2021 · 2021
Closest in time.
Edward J. Oughton. 2021 · 2021
Closest in time.
Carbon emissions and large neural network training
David Patterson, Joseph Gonzalez, Quoc Le, Chen Liang, Lluis-Miquel Munguia, Daniel Rothchild, David So, Maud Texier, and Jeff Dean. 2021 · 2021
Closest in time.
The curious case of hallucinations in neural machine translation
Vikas Raunak, Arul Menezes, and Marcin Junczys-Dowmunt. 2021 · 2021
Closest in time.
We need to talk about random splits
Anders Søgaard, Sebastian Ebert, Jasmijn Bastings, and Katja Filippova. 2021 · 2021
Closest in time.
Domain-specific MT for low-resource languages: The case of bambara-french
Allahsera Auguste Tapo, Michael Leventhal, Sarah Luger, Christopher M. Homan, and Marcos Zampieri. 2021 · 2021
Closest in time.