Parallel Corpus Filtering via Pre-trained Language Models

Understand

Web-crawled data provides a good source of parallel corpora for training machine translation models.

• It is automatically obtained, but extremely noisy, and recent work shows that neural machine translation systems are more sensitive to noise than traditional statistical machine translation methods.
• In this paper, we propose a novel approach to filter out noisy sentence pairs from web-crawled corpora via pre-trained language models.
• We measure sentence parallelism by leveraging the multilingual capability of BERT and use the Generative Pre-training (GPT) language model as a domain filter to balance data domains.