Emergent Communication Pretraining for Few-Shot Machine Translation

Understand

While state-of-the-art models that rely upon massively multilingual pretrained encoders achieve sample efficiency in downstream applications, they still require abundant amounts of unlabelled text.

• Nevertheless, most of the world's languages lack such resources.
• Hence, we investigate a more radical form of unsupervised knowledge transfer in the absence of linguistic data.
• In particular, for the first time we pretrain neural networks via emergent communication from referential games.