Multimodal Machine Translation with Reinforcement Learning

Understand

Multimodal machine translation is one of the applications that integrates computer vision and language processing.

• It is a unique task given that in the field of machine translation, many state-of-the-arts algorithms still only employ textual information.
• In this work, we explore the effectiveness of reinforcement learning in multimodal machine translation.
• We present a novel algorithm based on the Advantage Actor-Critic (A2C) algorithm that specifically cater to the multimodal machine translation task of the EMNLP 2018 Third Conference on Machine Translation (WMT18).