Transfer Fine-Tuning: A BERT Case Study

Understand

A semantic equivalence assessment is defined as a task that assesses semantic equivalence in a sentence pair by binary judgment (i.e., paraphrase identification) or grading (i.e., semantic textual similarity measurement).

• It constitutes a set of tasks crucial for research on natural language understanding.
• Recently, BERT realized a breakthrough in sentence representation learning (Devlin et al., 2019), which is broadly transferable to various NLP tasks.
• While BERT's performance improves by increasing its model size, the required computational power is an obstacle preventing practical applications from adopting the technology.