Mixup-Transformer: Dynamic Data Augmentation for NLP Tasks

Understand

Mixup is the latest data augmentation technique that linearly interpolates input examples and the corresponding labels.

• It has shown strong effectiveness in image classification by interpolating images at the pixel level.
• Inspired by this line of research, in this paper, we explore i) how to apply mixup to natural language processing tasks since text data can hardly be mixed in the raw format; ii) if mixup is still effective in transformer-based learning models, e.g., BERT.
• To achieve the goal, we incorporate mixup to transformer-based pre-trained architecture, named "mixup-transformer", for a wide range of NLP tasks while keeping the whole end-to-end training system.