On the Effective Use of Pretraining for Natural Language Inference

Understand

Neural networks have excelled at many NLP tasks, but there remain open questions about the performance of pretrained distributed word representations and their interaction with weight initialization and other hyperparameters.

• We address these questions empirically using attention-based sequence-to-sequence models for natural language inference (NLI).
• Specifically, we compare three types of embeddings: random, pretrained (GloVe, word2vec), and retrofitted (pretrained plus WordNet information).
• We show that pretrained embeddings outperform both random and retrofitted ones in a large NLI corpus.