MixUp Training Leads to Reduced Overfitting and Improved Calibration for the Transformer Architecture

Built on

Augmenting data with mixup for sentence classification: An empirical study
Original
Hongyu Guo, Yongyi Mao, and Richong Zhang. 2019 · 1905
Earlier work this paper cites.
Huggingface’s transformers: State-of-the-art natural language processing
Original
Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, R’emi Louf, Morgan Funtowicz, and Jamie Brew. 2019 · 1910
Earlier work this paper cites.
Learning word vectors for sentiment analysis
Andrew L. Maas, Raymond E. Daly, Peter T. Pham, Dan Huang, Andrew Y. Ng, and Christopher Potts. 2011 · 2011
Earlier work this paper cites.
On calibration of modern neural networks
Original
Chuan Guo, Geoff Pleiss, Yu Sun, and Kilian Q. Weinberger. 2017 · 2017
Earlier work this paper cites.
Character-level convolutional networks for text classification
Original
Xiang Zhang, Junbo Zhao, and Yann LeCun. 2015 · 2017
Earlier work this paper cites.

Glue: A multi-task benchmark and analysis platform for natural language understanding
Alex Wang, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel Bowman. 2018 · 2018
Cited alongside, same era.
mixup: Beyond empirical risk minimization
Original
Hongyi Zhang, Moustapha Cisse, Yann N. Dauphin, and David Lopez-Paz. 2018 · 2018
Cited alongside, same era.
Bert: Pre-training of deep bidirectional transformers for language understanding
Original
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019 · 2019
Cited alongside, same era.
On mixup training: Improved calibration and predictive uncertainty for deep neural networks
Original
Sunil Thulasidasan, Gopinath Chennupati, Jeff Bilmes, Tanmoy Bhattacharya, and Sarah Michalak. 2019 · 2019
Cited alongside, same era.

Manifold mixup: Better representations by interpolating hidden states
Original
Vikas Verma, Alex Lamb, Christopher Beckham, Amir Najafi, Ioannis Mitliagkas, Aaron Courville, David Lopez-Paz, and Yoshua Bengio. 2019 · 2019
Later among the works it cites.
Superglue: A stickier benchmark for general-purpose language understanding systems
Original
Alex Wang, Yada Pruksachatkun, Nikita Nangia, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel Bowman. 2019 · 2019
Later among the works it cites.
Calibration of pre-trained transformers
Original
Shrey Desai and Greg Durrett. 2020 · 2020
Later among the works it cites.
Mixup-transformer: Dynamic data augmentation for nlp tasks
Original
Lichao Sun, Congying Xia, Wenpeng Yin, Tingtin Liang, Philip S. Yu, and Lifang He. 2020 · 2020
Later among the works it cites.