Fetching the paper…
Reading the bibliography…
A multilingual tokenizer is a fundamental component of multilingual neural machine translation.
Massively multilingual neural machine translation in the wild: Findings and challenges
Arivazhagan, N., Bapna, A., Firat, O., Lepikhin, D., Johnson, M., Krikun, M., Chen, M. X., Cao, Y., Foster, G., Cherry, C., et al. (2019) · 1907
Earlier work this paper cites.
Statistical Power Analysis for the Behavioral Sciences
Cohen, J. (1988) · 1988
Earlier work this paper cites.
Bleu: a method for automatic evaluation of machine translation
Papineni, K., Roukos, S., Ward, T., and Zhu, W.-J. (2002) · 2002
Earlier work this paper cites.
Adv-bert: Bert is not robust on misspellings! generating nature adversarial samples on bert
Sun, L., Hashimoto, K., Yin, W., Asai, A., Li, J., Yu, P., and Xiong, C. (2020) · 2003
Earlier work this paper cites.
Moses: Open source toolkit for statistical machine translation
Koehn, P., Hoang, H., Birch, A., Callison-Burch, C., Federico, M., Bertoldi, N., Cowan, B., Shen, W., Moran, C., Zens, R., et al. (2007) · 2007
Earlier work this paper cites.
Japanese and korean voice search
Schuster, M. and Nakajima, K. (2012) · 2012
Earlier work this paper cites.
Adam: A method for stochastic optimization
Kingma, D. P. and Ba, J. (2015) · 2015
Earlier work this paper cites.
chrF: character n-gram F-score for automatic MT evaluation
Popović, M. (2015) · 2015
Earlier work this paper cites.
A character-level decoder without explicit segmentation for neural machine translation
Chung, J., Cho, K., and Bengio, Y. (2016) · 2016
Earlier work this paper cites.
Multi-way, multilingual neural machine translation with a shared attention mechanism
Firat, O., Cho, K., and Bengio, Y. (2016) · 2016
Earlier work this paper cites.
Neural machine translation of rare words with subword units
Sennrich, R., Haddow, B., and Birch, A. (2016) · 2016
Earlier work this paper cites.
Google’s multilingual neural machine translation system: Enabling zero-shot translation
Johnson, M., Schuster, M., Le, Q., Krikun, M., Wu, Y., Chen, Z., Thorat, N., Viégas, F., Wattenberg, M., Corrado, G., et al. (2017) · 2017
Earlier work this paper cites.
Six challenges for neural machine translation
Koehn, P. and Knowles, R. (2017) · 2017
Cited alongside, same era.
Fully character-level neural machine translation without explicit segmentation
Lee, J., Cho, K., and Hofmann, T. (2017) · 2017
Cited alongside, same era.
Attention is all you need
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, Ł., and Polosukhin, I. (2017) · 2017
Cited alongside, same era.
How much does tokenization affect neural machine translation?
Domingo, M., Garcıa-Martınez, M., Helle, A., Casacuberta, F., and Herranz, M. (2018) · 2018
Cited alongside, same era.
On the relation between linguistic typology and (limitations of) multilingual language modeling
Gerz, D., Vulić, I., Ponti, E. M., Reichart, R., and Korhonen, A. (2018) · 2018
Cited alongside, same era.
Byte pair encoding is suboptimal for language model pretraining
Bostrom, K. and Durrett, G. (2020) · 2020
Later among the works it cites.
Improving multilingual models with language-clustered vocabularies
Chung, H. W., Garrette, D., Tan, K. C., and Riesa, J. (2020) · 2020
Later among the works it cites.
Unsupervised cross-lingual representation learning at scale
Conneau, A., Khandelwal, K., Goyal, N., Chaudhary, V., Wenzek, G., Guzmán, F., Grave, É., Ott, M., Zettlemoyer, L., and Stoyanov, V. (2020) · 2020
Later among the works it cites.
Finding the optimal vocabulary size for neural machine translation
Gowda, T. and May, J. (2020) · 2020
Later among the works it cites.
Multilingual denoising pre-training for neural machine translation
Liu, Y., Gu, J., Goyal, N., Li, X., Edunov, S., Ghazvininejad, M., Lewis, M., and Zettlemoyer, L. (2020) · 2020
Later among the works it cites.
The FLORES-101 evaluation benchmark for low-resource and multilingual machine translation
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Kudo, T. (2018) · 2018
Cited alongside, same era.
Sentencepiece: A simple and language independent subword tokenizer and detokenizer for neural text processing
Kudo, T. and Richardson, J. (2018) · 2018
Cited alongside, same era.
A call for clarity in reporting BLEU scores
Post, M. (2018) · 2018
Cited alongside, same era.
Cross-lingual language model pretraining
Conneau, A. and Lample, G. (2019) · 2019
Cited alongside, same era.
Bert: Pre-training of deep bidirectional transformers for language understanding
Devlin, J., Chang, M.-W., Lee, K., and Toutanova, K. (2019) · 2019
Cited alongside, same era.
A call for prudent choice of subword merge operations in neural machine translation
Ding, S., Renduchintala, A., and Duh, K. (2019) · 2019
Cited alongside, same era.
Ács, J. (2019) · 2019
Cited alongside, same era.
Goyal, N., Gao, C., Chaudhary, V., Chen, P., Wenzek, G., Ju, D., Krishnan, S., Ranzato, M., Guzmán, F., and Fan, A. (2021) · 2021
Later among the works it cites.
How good is your tokenizer? on the monolingual performance of multilingual language models
Rust, P., Pfeiffer, J., Vulić, I., Ruder, S., and Gurevych, I. (2021) · 2021
Later among the works it cites.
Robust open-vocabulary translation from visual text representations
Salesky, E., Etter, D., and Post, M. (2021) · 2021
Later among the works it cites.
Charformer: Fast character transformers via gradient-based subword tokenization
Tay, Y., Tran, V. Q., Ruder, S., Gupta, J., Chung, H. W., Bahri, D., Qin, Z., Baumgartner, S., Yu, C., and Metzler, D. (2021) · 2021
Later among the works it cites.
Vocabulary learning via optimal transport for neural machine translation
Xu, J., Zhou, H., Gan, C., Zheng, Z., and Li, L. (2021) · 2021
Later among the works it cites.
mt5: A massively multilingual pre-trained text-to-text transformer
Xue, L., Constant, N., Roberts, A., Kale, M., Al-Rfou, R., Siddhant, A., Barua, A., and Raffel, C. (2021b) · 2021
Later among the works it cites.
Canine: Pre-training an efficient tokenization-free encoder for language representation
Clark, J. H., Garrette, D., Turc, I., and Wieting, J. (2022) · 2022
Closest in time.