How Transferable are Neural Networks in NLP Applications?

Understand

Transfer learning is aimed to make use of valuable knowledge in a source domain to help model performance in a target domain.

• It is particularly important to neural networks, which are very likely to be overfitting.
• In some fields like image processing, many studies have shown the effectiveness of neural network-based transfer learning.
• For neural NLP, however, existing studies have only casually applied transfer learning, and conclusions are inconsistent.