Fetching the paper…

Reading the bibliography…

2019

On Leveraging the Visual Modality for Neural Machine Translation

Raunak, Vikas, Choe, Sang Keun, Lu, Quanyang et al.

Original
PDF
Post on X

Understand

Leveraging the visual modality effectively for Neural Machine Translation (NMT) remains an open problem in computational linguistics.

Recently, Caglayan et al.
posit that the observed gains are limited mainly due to the very simple, short, repetitive sentences of the Multi30k dataset (the only multimodal MT dataset available at the time), which renders the source text sufficient for context.
In this work, we further investigate this hypothesis on a new large scale multimodal Machine Translation (MMT) dataset, How2, which has 1.57 times longer mean sentence length than Multi30k and no repetition.

On Leveraging the Visual Modality for Neural Machine Translation · Around

Built on

Improving sequence-to-sequence learning via optimal transport
Original
Liqun Chen, Yizhe Zhang, Ruiyi Zhang, Chenyang Tao, Zhe Gan, Haichao Zhang, Bai Li, Dinghan Shen, Changyou Chen, and Lawrence Carin. 2019 · 1901
Earlier work this paper cites.
Probing the need for visual context in multimodal machine translation
Original
Ozan Caglayan, Pranava Madhyastha, Lucia Specia, and Loïc Barrault. 2019 · 1903
Earlier work this paper cites.
Symbol interdependency in symbolic and embodied cognition
Max M Louwerse. 2011 · 2011
Earlier work this paper cites.
Is this a wampimuk? cross-modal mapping between distributional semantics and the visual world
Angeliki Lazaridou, Elia Bruni, and Marco Baroni. 2014 · 2014
Earlier work this paper cites.
Sequence to sequence learning with neural networks
Ilya Sutskever, Oriol Vinyals, and Quoc V Le. 2014 · 2014
Earlier work this paper cites.

Similar

Effective approaches to attention-based neural machine translation
Original
Minh-Thang Luong, Hieu Pham, and Christopher D Manning. 2015 · 2015
Cited alongside, same era.
Neural machine translation of rare words with subword units
Original
Rico Sennrich, Barry Haddow, and Alexandra Birch. 2015 · 2015
Cited alongside, same era.
Multimodal attention for neural machine translation
Original
Ozan Caglayan, Loïc Barrault, and Fethi Bougares. 2016 · 2016
Cited alongside, same era.
Attention-based multimodal neural machine translation
Po-Yao Huang, Frederick Liu, Sz-Rung Shiang, Jean Oh, and Chris Dyer. 2016 · 2016
Cited alongside, same era.
Google’s neural machine translation system: Bridging the gap between human and machine translation
Original

Then

Imagination improves multimodal translation
Original
Desmond Elliott and Akos Kádár. 2017 · 2017
Later among the works it cites.
Attention strategies for multi-source sequence-to-sequence learning
Original
Jindřich Libovickỳ and Jindřich Helcl. 2017 · 2017
Later among the works it cites.
Aggregated residual transformations for deep neural networks
Saining Xie, Ross Girshick, Piotr Dollár, Zhuowen Tu, and Kaiming He. 2017 · 2017
Later among the works it cites.
Sentencepiece: A simple and language independent subword tokenizer and detokenizer for neural text processing
Original
Taku Kudo and John Richardson. 2018 · 2018
Later among the works it cites.
All-but-the-top: Simple and effective postprocessing for word representations
Jiaqi Mu and Pramod Viswanath. 2018 · 2018

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Yonghui Wu, Mike Schuster, Zhifeng Chen, Quoc V Le, Mohammad Norouzi, Wolfgang Macherey, Maxim Krikun, Yuan Cao, Qin Gao, Klaus Macherey, et al. 2016 · 2016

Cited alongside, same era.

How2: A large-scale dataset for multimodal language understanding

Ramon Sanabria, Ozan Caglayan, Shruti Palaskar, Desmond Elliott, Loïc Barrault, Lucia Specia, and Florian Metze. 2018a

Cited in the paper.

How2: A large-scale dataset for multimodal language understanding

Ramon Sanabria, Ozan Caglayan, Shruti Palaskar, Desmond Elliott, Loïc Barrault, Lucia Specia, and Florian Metze. 2018b

Cited in the paper.

Later among the works it cites.