2018

Denoising Neural Machine Translation Training with Trusted Data and Online Data Selection

Wang, Wei, Watanabe, Taro, Hughes, Macduff et al.

Understand

Measuring domain relevance of data and identifying or selecting well-fit domain data for machine translation (MT) is a well-studied topic, but denoising is not yet.

Denoising is concerned with a different type of data quality and tries to reduce the negative impact of data noise on MT training, in particular, neural MT (NMT) training.
This paper generalizes methods for measuring and selecting data for domain MT and applies them to denoising NMT training.
The proposed approach uses trusted data and a denoising curriculum realized by online data selection.

Built on

Statistical significance tests for machine translation evaluation
Philipp Koehn. 2004 · 2004
Earlier work this paper cites.
Curriculum learning
Yoshua Bengio, Jérôme Louradour, Ronan Collobert, and Jason Weston. 2009 · 2009
Earlier work this paper cites.
Discriminative corpus weight estimation for machine translation
Spyros Matsoukas, Antti-Veikko I. Rosti, and Bing Zhang. 2009 · 2009
Earlier work this paper cites.
Self-paced learning for latent variable models
M. P. Kumar, Benjamin Packer, and Daphne Koller. 2010 · 2010
Earlier work this paper cites.
Intelligent selection of language model training data
Robert C. Moore and William Lewis. 2010 · 2010
Earlier work this paper cites.
Active learning literature survey
Burr Settles. 2010 · 2010
Earlier work this paper cites.
Domain adaptation via pseudo in-domain data selection
Amittai Axelrod, Xiaodong He, and Jianfeng Gao. 2011 · 2011
Earlier work this paper cites.
Combining translation and language model scoring for domain-specific data filtering
Saab Mansour, Joern Wuebker, and Hermann Ney. 2011 · 2011
Earlier work this paper cites.

Similar

Learning with noisy labels
Nagarajan Natarajan, Inderjit S Dhillon, Pradeep K Ravikumar, and Ambuj Tewari. 2013 · 2013
Cited alongside, same era.
Semi-supervised convolutional networks for translation adaptation with tiny amount of in-domain data
Boxing Chen and Fei Huang. 2016 · 2016
Cited alongside, same era.
Bilingual methods for adaptive training data selection for machine translation
Boxing Chen, Roland Kuhn, George Foster, Colin Cherry, and Fei Huang. 2016 · 2016
Cited alongside, same era.
Google’s neural machine translation system: Bridging the gap between human and machine translation
Original
Yonghui Wu, Mike Schuster, Zhifeng Chen, Quoc V Le, Mohammad Norouzi, Wolfgang Macherey, Maxim Krikun, Yuan Cao, Qin Gao, Klaus Macherey, Jeff Klingner, Apurva Shah, Melvin Johnson, Xiaobing Liu, Łukasz Kaiser, Stephan Gouws, Yoshikiyo Kato, Taku Kudo, Hideto Kazawa, Keith Stevens, George Kurian, Nishant Patil, Wei Wang, Cliff Young, Jason Smith, Jason Riesa, Alex Rudnick, Oriol Vinyals, Greg Corrado, Macduff Hughes, and Jeffrey Dean. 2016 · 2016
Cited alongside, same era.
Effective domain mixing for neural machine translation
Denny Britz, Quoc Le, and Reid Pryzant. 2017 · 2017
Cited alongside, same era.
Convolutional sequence to sequence learning
Original
Jonas Gehring, Michael Auli, David Grangier, Denis Yarats, and Yann N. Dauphin. 2017 · 2017
Cited alongside, same era.
Unsupervised machine translation using monolingual corpora only
Original
Guillaume Lample, Ludovic Denoyer, and Marc’Aurelio Ranzato. 2017 · 2017
Cited alongside, same era.
Learning from Noisy Data in Statistical Machine Translation
Mohammed Mediani. 2017 · 2017
Cited alongside, same era.

Then

Neural machine translation training in a multi-domain scenario
Original
Hassan Sajjad, Nadir Durrani, Fahim Dalvi, Yonatan Belinkov, and Stephan Vogel. 2017 · 2017
Later among the works it cites.
Attention is all you need
Original
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, , and Illia Polosukhin. 2017 · 2017
Later among the works it cites.
Instance weighting for neural machine translation domain adaptation
Rui Wang, Masao Utiyama, Lemao Liu, Kehai Chen, and Eiichiro Sumita. 2017 · 2017
Later among the works it cites.
Dynamic data selection for neural machine transaltion
Marlies van der Wees, Arianna Bisazza, and Christof Monz. 2017 · 2017
Later among the works it cites.
The best of both worlds: Combining recent advances in neural machine translation
Mia Xu Chen, Orhan Firat, Ankur Bapna, Melvin Johnson, Wolfgang Macherey, George Foster, Llion Jones, Niki Parmar, Mike Schuster, Zhifeng Chen, Yonghui Wu, and Macduff Hughes. 2018 · 2018
Closest in time.
On the impact of various types of noise on neural machine translation
Original
Huda Khayrallah and Philipp Koehn. 2018 · 2018
Closest in time.
A call for clarity in reporting bleu scores
Original
Matt Post. 2018 · 2018
Closest in time.
Identifying semantic divergences in parallel text without annotations
Yogarshi Vyas, Xing Niu, and Marine Carpuat. 2018 · 2018
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…