Is Prompt-Based Finetuning Always Better than Vanilla Finetuning? Insights from Cross-Lingual Language Understanding

Understand

Multilingual pretrained language models (MPLMs) have demonstrated substantial performance improvements in zero-shot cross-lingual transfer across various natural language understanding tasks by finetuning MPLMs on task-specific labelled data of a source language (e.g.

• English) and evaluating on a wide range of target languages.
• Recent studies show that prompt-based finetuning surpasses regular finetuning in few-shot scenarios.
• However, the exploration of prompt-based learning in multilingual tasks remains limited.