2017

Towards Neural Phrase-based Machine Translation

Huang, Po-Sen, Wang, Chong, Huang, Sitao et al.

Understand

In this paper, we present Neural Phrase-based Machine Translation (NPMT).

Our method explicitly models the phrase structures in output sequences using Sleep-WAke Networks (SWAN), a recently proposed segmentation-based sequence modeling method.
To mitigate the monotonic alignment requirement of SWAN, we introduce a new layer to perform (soft) local reordering of input sequences.
Different from existing neural machine translation (NMT) approaches, NPMT does not use attention-based decoding mechanisms.

Built on

The mathematics of statistical machine translation: Parameter estimation
Peter F Brown, Vincent J Della Pietra, Stephen A Della Pietra, and Robert L Mercer · 1993
Earlier work this paper cites.
Long short-term memory
Sepp Hochreiter and Jürgen Schmidhuber · 1997
Earlier work this paper cites.
BLEU: a method for automatic evaluation of machine translation
Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu · 2002
Earlier work this paper cites.
Statistical phrase-based translation
Philipp Koehn, Franz Josef Och, and Daniel Marcu · 2003
Earlier work this paper cites.
Statistical machine translation
Adam Lopez · 2008
Earlier work this paper cites.
Statistical Machine Translation
Philipp Koehn · 2009
Earlier work this paper cites.
Scalable modified Kneser-Ney language model estimation
Kenneth Heafield, Ivan Pouzyrevsky, Jonathan H. Clark, and Philipp Koehn · 2013
Earlier work this paper cites.
Report on the 11th IWSLT evaluation campaign, IWSLT 2014
Mauro Cettolo, Jan Niehues, Sebastian Stüker, Luisa Bentivogli, and Marcello Federico · 2014
Earlier work this paper cites.
Empirical evaluation of gated recurrent neural networks on sequence modeling
Original
Junyoung Chung, Caglar Gulcehre, KyungHyun Cho, and Yoshua Bengio · 2014
Earlier work this paper cites.
Deep speech: Scaling up end-to-end speech recognition
Original
Awni Hannun, Carl Case, Jared Casper, Bryan Catanzaro, Greg Diamos, Erich Elsen, Ryan Prenger, Sanjeev Satheesh, Shubho Sengupta, Adam Coates, et al · 2014
Earlier work this paper cites.
Adam: A method for stochastic optimization
Original
Diederik Kingma and Jimmy Ba · 2014
Earlier work this paper cites.

Similar

First-pass large vocabulary continuous speech recognition using bi-directional recurrent DNNs
Original
Andrew L. Maas, Awni Y. Hannun, Daniel Jurafsky, and Andrew Y. Ng · 2014
Cited alongside, same era.
Sequence to sequence learning with neural networks
Ilya Sutskever, Oriol Vinyals, and Quoc V Le · 2014
Cited alongside, same era.
Neural machine translation by jointly learning to align and translate
Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio · 2015
Cited alongside, same era.
The IWSLT 2015 evaluation campaign
Mauro Cettolo, Jan Niehues, Sebastian Stüker, Luisa Bentivogli, Roldano Cattoni, and Marcello Federico · 2015
Cited alongside, same era.
Stanford neural machine translation systems for spoken language domains
Minh-Thang Luong and Christopher D Manning · 2015
Cited alongside, same era.
Effective approaches to attention-based neural machine translation
Original
Minh-Thang Luong, Hieu Pham, and Christopher D Manning · 2015
Cited alongside, same era.
Sequence level training with recurrent neural networks
Original
Marc’Aurelio Ranzato, Sumit Chopra, Michael Auli, and Wojciech Zaremba · 2015
Cited alongside, same era.
Neural machine translation with external phrase memory
Original
Yaohua Tang, Fandong Meng, Zhengdong Lu, Hang Li, and Philip LH Yu · 2016
Cited alongside, same era.
Sequence-to-sequence learning as beam-search optimization
Original
Sam Wiseman and Alexander M. Rush · 2016
Cited alongside, same era.
Google’s neural machine translation system: Bridging the gap between human and machine translation
Original
Yonghui Wu, Mike Schuster, Zhifeng Chen, Quoc V Le, Mohammad Norouzi, Wolfgang Macherey, Maxim Krikun, Yuan Cao, Qin Gao, Klaus Macherey, et al · 2016
Cited alongside, same era.
Online segment to segment neural transduction
Lei Yu, Jan Buys, and Phil Blunsom · 2016
Cited alongside, same era.
Sequence modeling via segmentations
Chong Wang, Yining Wang, Po-Sen Huang, Abdelrahman Mohamed, Dengyong Zhou, and Li Deng
Cited in the paper.

Then

Deep recurrent models with fast-forward connections for neural machine translation
Jie Zhou, Ying Cao, Xuguang Wang, Peng Li, and Wei Xu · 2016
Later among the works it cites.
An actor-critic algorithm for sequence prediction
Dzmitry Bahdanau, Philemon Brakel, Kelvin Xu, Anirudh Goyal, Ryan Lowe, Joelle Pineau, Aaron C. Courville, and Yoshua Bengio · 2017
Closest in time.
Neural machine translation leveraging phrase-based models in a hybrid search
Leonard Dahlmann, Evgeny Matusov, Pavel Petrushkov, and Shahram Khadivi · 2017
Closest in time.
Language modeling with gated convolutional networks
Yann N Dauphin, Angela Fan, Michael Auli, and David Grangier · 2017
Closest in time.
Convolutional sequence to sequence learning
Jonas Gehring, Michael Auli, David Grangier, Denis Yarats, and Yann N Dauphin · 2017
Closest in time.
Online and linear-time attention by enforcing monotonic alignments
Colin Raffel, Thang Luong, Peter J Liu, Ron J Weiss, and Douglas Eck · 2017
Closest in time.
Attention is all you need
Original
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin · 2017
Closest in time.
Translating phrases in neural machine translation
Xing Wang, Zhaopeng Tu, Deyi Xiong, and Min Zhang · 2017
Closest in time.
The neural noisy channel
Lei Yu, Phil Blunsom, Chris Dyer, Edward Grefenstette, and Tomas Kocisky · 2017
Closest in time.
Non-autoregressive neural machine translation
Jiatao Gu, James Bradbury, Caiming Xiong, Victor O.K. Li, and Richard Socher · 2018
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…