2017

Synthetic Data for Neural Machine Translation of Spoken-Dialects

Hassan, Hany, Elaraby, Mostafa, Tawfik, Ahmed

Understand

In this paper, we introduce a novel approach to generate synthetic data for training Neural Machine Translation systems.

The proposed approach transforms a given parallel corpus between a written language and a target language to a parallel corpus between a spoken dialect variant and the target language.
Our approach is language independent and can be used to generate data for any variant of the source language such as slang or spoken dialect or even for a different language that is closely related to the source language.
The proposed approach is based on local embedding projection of distributed representations which utilizes monolingual embeddings to transform parallel data across language variants.

Built on

Sam T Roweis and Lawrence K Saul. “Nonlinear dimensionality reduction by locally linear embedding.“, In Science, 2000
2000
Earlier work this paper cites.
Lisa Lim, L., Deterding, D., “Singapore English“, Journal of the International Phonetic Association, 2010
2010
Earlier work this paper cites.
“Composition in distributional models of semantics.“, In Cognitive science, 2010
Jeff Mitchell and Mirella Lapata. 2010 · 2010
Earlier work this paper cites.
Al-Shareef, S. and Hain, T., “An investigation in speech recognition for colloquial Arabic“, INTERSPEECH 2011
2011
Earlier work this paper cites.
Mikolov, T.,Sutskever, I., Chen, k., Corrado, G., Dean, J., “Distributed representations of words and phrases and their compositionality.“, In NIPS 2013
2013
Earlier work this paper cites.
Saluja, A., Hassan, H., Toutanova, K., Quirk, C., “Graph-based semi-supervised learning of translation models from monolingual data.“, In ACL 2014
2014
Earlier work this paper cites.

Similar

Ilya Sutskever, I., Vinyals, O., and Le, Q., “ Sequence to sequence learning with neural networks.“, In Advances in Neural Information Processing Systems 2014
2014
Cited alongside, same era.
Bahdanau, D., Cho, K., Bengio, Y., “Neural machine translation by jointly learning to align and translate“, ICLR 2015
2015
Cited alongside, same era.
Zhao, K., Hassan, H., Auli, M., “Learning translation models from monolingual continuous representations.“, In NAACL HLT 2015
2015
Cited alongside, same era.
Sajjad, H., Durrani, N., Guzman, F., Nakov, P., Abdelali, A., Vogel, S., Salloum, W., ElKholy, A., Habash, N. “Egyptian arabic to english statistical machine translation system for NIST openmt’2015.“ In CoRR
Original
2015
Cited alongside, same era.
Luong, M., Pham, H., Manning, D., “Effective approaches to attention-based neural machine translation“, in EMNLP 2015
2015
Cited alongside, same era.
Rico Sennrich, Barry Haddow, and Alexandra Birch. “Neural machine translation of rare words with subword units.“, In ACL 2016
2016
Cited alongside, same era.
Wu, Y., Schuster, M., Chen, Z., Le, Q., Norouzi, M., Macherey, W., Krikun, M., Cao, Y., Gao, Q., Macherey, K., Klingner, J., Shah, A., Johnson, M., Liu, X. , Kaiser, L., Gouws, S., Kato, Y., Kudo, T., Kazawa, H., Stevens, K., Kurian, G., Patil, N., Wang,W., Young, C., Smith, J., Riesa, J., Rudnick, A., Vinyals, O., Corrado, G., Hughes, M., Dean, J., “Google’s neural machine translation system: Bridging the gap between human and machine translation.“, https://arxiv.org/abs/1609.08144
Original
Cited in the paper.
Zbib, R., Malchiodi, E., Devlin, J., Stallard, D., Matsoukas. S., Schwartz, R., Makhoul, J., Zaidan, O., Callison-Burch, C., “Machine translation of arabic dialects.“ , In HLT/NAACL 201
Cited in the paper.
Durrani, N., Al-Onaizan, Y., Ittycheriah, A., “Improving Egyptian-to-English SMT by Mapping Egyptian into MSA“, Springer Berlin Heidelberg, Berlin, Heidelberg, pages 271–282
Cited in the paper.
Klementiev, A., Irvine,A., , Callison-Burch, C., Yarowsky, D., “Toward statistical machine translation without parallel corpora.“, In Proceedings of EACL ’12
Cited in the paper.
Indyk, P., and Motwani ,R., “Approximate nearest neighbors: towards removing the curse of dimensionality.“ In Proceedings of the thirtieth annual ACM symposium on Theory, of computing
Cited in the paper.
Kingma, D., and Ba, J., “Adam: A method for stochastic optimization“, http://arxiv.org/abs/1412.6980
Original
Cited in the paper.

Then

Niu, X. and Carpuat, M., “The umd machine translation systems at iwslt 2016: English-to-french translation of speech transcripts.“, In IWSLT, 2016
2016
Later among the works it cites.
Rico Sennrich, R. , Haddow, B., Birch, A., “ Improving neural machine translation models with monolingual data.“, In ACL 2016
2016
Later among the works it cites.
Hyvönen, V., Pitkänen, T., Tasoulis, T., Jaasaari, E., Tuomainen, R., Wang, L., Corander, J. Roos, T., “Fast nearest neighbor search through sparse random projections and voting“, In 2016 IEEE International Conference on Big Data, BigData
2016
Later among the works it cites.
Firat, O., Cho, K., Bengio, Y., “Multi-way, multilingual neural machine translation with a shared attention mechanism“ ,HLT-NAACL 2016
2016
Later among the works it cites.
Rico Sennrich, R., Haddow, B. Birch, A. “Neural machine translation of rare words with subword units.“, In ACL 2016
2016
Later among the works it cites.
Devlin, J., “Sharp Models on Dull Hardware: Fast and Accurate Neural Machine Translation Decoding on the CPU“, EMNLP 2017: 2810-2815
2017
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…