Investigating Transfer Learning in Multilingual Pre-trained Language Models through Chinese Natural Language Inference

Understand

Multilingual transformers (XLM, mT5) have been shown to have remarkable transfer skills in zero-shot settings.

• Most transfer studies, however, rely on automatically translated resources (XNLI, XQuAD), making it hard to discern the particular linguistic knowledge that is being transferred, and the role of expert annotated monolingual datasets when developing task-specific models.
• We investigate the cross-lingual transfer abilities of XLM-R for Chinese and English natural language inference (NLI), with a focus on the recent large-scale Chinese dataset OCNLI.
• To better understand linguistic transfer, we created 4 categories of challenge and adversarial tasks (totaling 17 new datasets) for Chinese that build on several well-known resources for English (e.g., HANS, NLI stress-tests).