Fine-Tuning Transformers: Vocabulary Transfer

Y. Liu, M. Ott, N. Goyal, J. Du, M. Joshi, D. Chen, O. Levy, M. Lewis, L. Zettlemoyer, V. Stoyanov, Roberta: A robustly optimized bert pretraining approach, arXiv preprint arXiv:1907.11692

Original

1907

Earlier work this paper cites.

C. Raffel, N. Shazeer, A. Roberts, K. Lee, S. Narang, M. Matena, Y. Zhou, W. Li, P. J. Liu, Exploring the limits of transfer learning with a unified text-to-text transformer, arXiv preprint arXiv:1910.10683

Original

1910

Earlier work this paper cites.

S. M. Lakew, M. Di Gangi, M. Federico, Controlling the output length of neural machine translation, arXiv preprint arXiv:1910.10408

Original

1910

Earlier work this paper cites.

P. Bojanowski, O. Celebi, T. Mikolov, E. Grave, A. Joulin, Updating pre-trained word vectors and text classifiers using monolingual alignment, arXiv preprint arXiv:1910.06241

Original

1910

Earlier work this paper cites.

P. Gage, A new algorithm for data compression, C Users Journal 12 (2) (1994) 23–38

1994

Earlier work this paper cites.

A. Chronopoulou, D. Stojanovski, V. Hangya, A. Fraser, The lmu munich system for the wmt 2020 unsupervised machine translation shared task, arXiv preprint arXiv:2010.13192

Original

2010

Earlier work this paper cites.

M. Schuster, K. Nakajima, Japanese and korean voice search, in: 2012 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), IEEE, 2012, pp. 5149–5152

2012

Earlier work this paper cites.

doi:10.18653/v1/P16-1162

R. Sennrich, B. Haddow, A. Birch, Neural machine translation of rare words with subword units , in: Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Association for Computational Linguistics, Berlin, Germany, 2016, pp. 1715–1725 · 2016

Cited alongside, same era.

A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, I. Polosukhin, Attention is all you need, in: Advances in neural information processing systems, 2017, pp. 5998–6008

2017

Cited alongside, same era.

T. Kudo, Subword regularization: Improving neural network translation models with multiple subword candidates, in: Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 2018, pp. 66–75

2018

Cited alongside, same era.

T. Kudo, J. Richardson, Sentencepiece: A simple and language independent subword tokenizer and detokenizer for neural text processing, in: Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, 2018, pp. 66–71

2018

K. Bostrom, G. Durrett, Byte pair encoding is suboptimal for language model pretraining, in: Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: Findings, 2020, pp. 4617–4624

2020

Later among the works it cites.

I. Provilkov, D. Emelianenko, E. Voita, Bpe-dropout: Simple and effective subword regularization, in: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, 2020, pp. 1882–1892

2020

Later among the works it cites.

A. F. Aji, N. Bogoychev, K. Heafield, R. Sennrich, In neural machine translation, what does transfer learning transfer?, in: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, 2020, pp. 7701–7710

2020

Later among the works it cites.

S. Sato, J. Sakuma, N. Yoshinaga, M. Toyoda, M. Kitsuregawa, Vocabulary adaptation for domain adaptation in neural machine translation, in: Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: Findings, 2020, pp. 4269–4279