Fetching the paper…

Reading the bibliography…

2019

Low Resource Text Classification with ULMFit and Backtranslation

Shleifer, Sam

Original
PDF
Post on X

Understand

In computer vision, virtually every state-of-the-art deep learning system is trained with data augmentation.

In text classification, however, data augmentation is less widely practiced because it must be performed before training and risks introducing label noise.
We augment the IMDB movie reviews dataset with examples generated by two families of techniques: random token perturbations introduced by Wei and Zou [2019] and backtranslation -- translating to a second language then back to English.
In low resource environments, backtranslation generates significant improvement on top of the state of-the-art ULMFit model.

Low Resource Text Classification with ULMFit and Backtranslation · Around

Built on

Learning word vectors for sentiment analysis
Andrew L. Maas, Raymond E. Daly, Peter T. Pham, Dan Huang, Andrew Y. Ng, and Christopher Potts · 2011
Earlier work this paper cites.
Improving neural machine translation models with monolingual data
Rico Sennrich, Barry Haddow, and Alexandra Birch · 2016
Earlier work this paper cites.
Pointer sentinel mixture models
Original
Stephen Merity, Caiming Xiong, James Bradbury, and Richard Socher · 2016
Earlier work this paper cites.
Virtual adversarial training for semi-supervised text classification
Takeru Miyato, Andrew M Dai, and Ian Goodfellow · 2016
Earlier work this paper cites.

Similar

Regularizing and optimizing lstm language models
Original
Stephen Merity, Nitish Shirish Keskar, and Richard Socher · 2017
Cited alongside, same era.
Universal language model fine-tuning for text classification
Jeremy Howard and Sebastian Ruder · 2018
Cited alongside, same era.
Understanding back-translation at scale
Original
Sergey Edunov, Myle Ott, Michael Auli, and David Grangier · 2018
Cited alongside, same era.

Then

Interpretable adversarial perturbation in input embedding space for text
Original
Motoki Sato, Jun Suzuki, Hiroyuki Shindo, and Yuji Matsumoto · 2018
Later among the works it cites.
Bert: Pre-training of deep bidirectional transformers for language understanding
Original
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova · 2018
Later among the works it cites.
Eda: Easy data augmentation techniques for boosting performance on text classification tasks, 2019
Jason W. Wei and Kai Zou · 2019
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…