TANDA: Transfer and Adapt Pre-Trained Transformer Models for Answer Sentence Selection

Understand

We propose TANDA, an effective technique for fine-tuning pre-trained Transformer models for natural language tasks.

• Specifically, we first transfer a pre-trained model into a model for a general task by fine-tuning it with a large and high-quality dataset.
• We then perform a second fine-tuning step to adapt the transferred model to the target domain.
• We demonstrate the benefits of our approach for answer sentence selection, which is a well-known inference task in Question Answering.