Real-Time Execution of Large-scale Language Models on Mobile

Huggingface’s transformers: State-of-the-art natural language processing

Original

Wolf, T.; Debut, L.; Sanh, V.; Chaumond, J.; Delangue, C.; Moi, A.; Cistac, P.; Rault, T.; Louf, R.; Funtowicz, M.; et al. 2019 · 1910

Earlier work this paper cites.

Very Deep Convolutional Networks for Large-Scale Image Recognition

Simonyan, K.; and Zisserman, A. 2014 · 2014

Earlier work this paper cites.

XSEDE: Accelerating Scientific Discovery

Towns, J.; Cockerill, T.; Dahan, M.; Foster, I.; Gaither, K.; Grimshaw, A.; Hazlewood, V.; Lathrop, S.; Lifka, D.; Peterson, G. D.; Roskies, R.; Scott, J. R.; and Wilkins-Diehr, N. 2014 · 2014

Earlier work this paper cites.

Aligning Books and Movies: Towards Story-like Visual Explanations by Watching Movies and Reading Books

Original

Zhu, Y.; Kiros, R.; Zemel, R.; Salakhutdinov, R.; Urtasun, R.; Torralba, A.; and Fidler, S. 2015 · 2015

Earlier work this paper cites.

https://www.tensorflow.org/mobile/tflite/

TensorFlow. 2017 · 2017

Earlier work this paper cites.

ProxylessNAS: Direct Neural Architecture Search on Target Task and Hardware

Cai, H.; Zhu, L.; and Han, S. 2018 · 2018

Cited alongside, same era.

BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding

Devlin, J.; Chang, M.-W.; Lee, K.; and Toutanova, K. 2018 · 2018

Cited alongside, same era.

MnasNet: Platform-Aware Neural Architecture Search for Mobile

Tan, M.; Chen, B.; Pang, R.; Vasudevan, V.; Sandler, M.; Howard, A.; and Le, Q. V. 2018 · 2018

Cited alongside, same era.

Glue: A multi-task benchmark and analysis platform for natural language understanding

Original

Wang, A.; Singh, A.; Michael, J.; Hill, F.; Levy, O.; and Bowman, S. R. 2018 · 2018

Cited alongside, same era.

FBNet: Hardware-Aware Efficient ConvNet Design via Differentiable Neural Architecture Search

Wu, B.; Dai, X.; Zhang, P.; Wang, Y.; Sun, F.; Wu, Y.; Tian, Y.; Vajda, P.; Jia, Y.; and Keutzer, K. 2018 · 2018

Cited alongside, same era.

RoBERTa: A Robustly Optimized BERT Pretraining Approach

Liu, Y.; Ott, M.; Goyal, N.; Du, J.; Joshi, M.; Chen, D.; Levy, O.; Lewis, M.; Zettlemoyer, L.; and Stoyanov, V. 2019 · 2019

Later among the works it cites.

Language Models are Unsupervised Multitask Learners

Radford, A.; Wu, J.; Child, R.; Luan, D.; Amodei, D.; and Sutskever, I. 2019 · 2019

Later among the works it cites.

DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter

Sanh, V.; Debut, L.; Chaumond, J.; and Wolf, T. 2019 · 2019

Later among the works it cites.

XLNet: Generalized Autoregressive Pretraining for Language Understanding

Yang, Z.; Dai, Z.; Yang, Y.; Carbonell, J.; Salakhutdinov, R.; and Le, Q. V. 2019 · 2019

Later among the works it cites.

MobileBERT: a Compact Task-Agnostic BERT for Resource-Limited Devices

Sun, Z.; Yu, H.; Song, X.; Liu, R.; Yang, Y.; and Zhou, D. 2020 · 2020

Closest in time.