Fetching the paper…
Reading the bibliography…
Multilingual Language Models (\MLLMs) such as mBERT, XLM, XLM-R, \textit{etc.} have emerged as a viable option for bringing the power of pretraining to a large number of languages.
Roberta: A robustly optimized bert pretraining approach
Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, M. Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019b · 1907
Earlier work this paper cites.
Unsupervised cross-lingual representation learning at scale
Alexis Conneau, Kartikay Khandelwal, Naman Goyal, Vishrav Chaudhary, Guillaume Wenzek, Francisco Guzmán, Edouard Grave, Myle Ott, Luke Zettlemoyer, and Veselin Stoyanov. 2019 · 1911
Earlier work this paper cites.
How language-neutral is multilingual bert?
Jindřich Libovickỳ, Rudolf Rosa, and Alexander Fraser. 2019 · 1911
Earlier work this paper cites.
Towards lingua franca named entity recognition with BERT
Taesun Moon, Parul Awasthy, Jian Ni, and Radu Florian. 2019 · 1912
Earlier work this paper cites.
Multilingual is not enough: BERT for finnish
Antti Virtanen, Jenna Kanerva, Rami Ilo, Jouni Luoma, Juhani Luotolahti, Tapio Salakoski, Filip Ginter, and Sampo Pyysalo. 2019 · 1912
Earlier work this paper cites.
Language universals
Joseph H Greenberg. 1966 · 1966
Earlier work this paper cites.
The mathematics of statistical machine translation: Parameter estimation
Peter F. Brown, Stephen A. Della Pietra, Vincent J. Della Pietra, and Robert L. Mercer. 1993 · 1993
Earlier work this paper cites.
Introduction to the CoNLL-2002 shared task: Language-independent named entity recognition
Erik F. Tjong Kim Sang. 2002 · 2002
Earlier work this paper cites.
Incorporating BERT into neural machine translation
Jinhua Zhu, Yingce Xia, Lijun Wu, Di He, Tao Qin, Wengang Zhou, Houqiang Li, and Tie-Yan Liu. 2020 · 2002
Earlier work this paper cites.
A systematic comparison of various statistical alignment models
Franz Josef Och and Hermann Ney. 2003 · 2003
Earlier work this paper cites.
Pre-trained models for natural language processing: A survey
Xipeng Qiu, Tianxiang Sun, Yige Xu, Yunfan Shao, Ning Dai, and Xuanjing Huang. 2020 · 2003
Earlier work this paper cites.
Introduction to the CoNLL-2003 shared task: Language-independent named entity recognition
Erik F. Tjong Kim Sang and Fien De Meulder. 2003 · 2003
Earlier work this paper cites.
The state and fate of linguistic diversity and inclusion in the nlp world
Pratik Joshi, Sebastin Santy, Amar Budhiraja, Kalika Bali, and Monojit Choudhury. 2020b · 2004
Earlier work this paper cites.
On the evaluation of contextual embeddings for zero-shot cross-lingual transfer learning
Phillip Keung, Yichao Lu, Julian Salazar, and Vikas Bhardwaj. 2020 · 2004
Earlier work this paper cites.
A study of cross-lingual ability and language-specific information in multilingual BERT
Chi-Liang Liu, Tsung-Yuan Hsu, Yung-Sung Chuang, and Hung-yi Lee. 2020a · 2004
Earlier work this paper cites.
Exploring fine-tuning techniques for pre-trained cross-lingual models via continual learning
Zihan Liu, Genta Indra Winata, Andrea Madotto, and Pascale Fung. 2020c · 2004
Earlier work this paper cites.
Udapter: Language adaptation for truly universal dependency parsing
Ahmet Üstün, Arianna Bisazza, Gosse Bouma, and Gertjan van Noord. 2020 · 2004
Earlier work this paper cites.
Language (technology) is power: A critical survey of" bias" in nlp
Su Lin Blodgett, Solon Barocas, Hal Daumé III, and Hanna Wallach. 2020 · 2005
Earlier work this paper cites.
Automatically constructing a corpus of sentential paraphrases
William B Dolan and Chris Brockett. 2005 · 2005
Earlier work this paper cites.
Anne Lauscher, Vinit Ravishankar, Ivan Vulic, and Goran Glavas. 2020 · 2005
Earlier work this paper cites.
Mono vs multilingual transformer-based models: a comparison across several language tasks
Diego de Vargas Feijó and Viviane Pereira Moreira. 2020 · 2007
Earlier work this paper cites.
Language-agnostic BERT sentence embedding
Fangxiaoyu Feng, Yinfei Yang, Daniel Cer, Naveen Arivazhagan, and Wei Wang. 2020 · 2007
Earlier work this paper cites.
Adapterhub: A framework for adapting transformers
Jonas Pfeiffer, Andreas Rücklé, Clifton Poth, Aishwarya Kamath, Ivan Vulic, Sebastian Ruder, Kyunghyun Cho, and Iryna Gurevych. 2020a · 2007
Earlier work this paper cites.
Inducing language-agnostic multilingual representations
Wei Zhao, Steffen Eger, Johannes Bjerva, and Isabelle Augenstein. 2020 · 2008
Earlier work this paper cites.
Rochelle Choenni and Ekaterina Shutova. 2020 · 2009
Earlier work this paper cites.
Efficient transformers: A survey
Yi Tay, Mostafa Dehghani, Dara Bahri, and Donald Metzler. 2020 · 2009
Earlier work this paper cites.
Benjamin Müller, Antonis Anastasopoulos, Benoît Sagot, and Djamé Seddah. 2020 · 2010
Earlier work this paper cites.
Interpretable multi-dataset evaluation for named entity recognition
Jinlan Fu, Pengfei Liu, and Graham Neubig. 2020 · 2011
Earlier work this paper cites.
Shuming Ma, Jian Yang, Haoyang Huang, Zewen Chi, Li Dong, Dongdong Zhang, Hany Hassan Awadalla, Alexandre Muzio, Akiko Eriguchi, Saksham Singhal, Xia Song, Arul Menezes, and Furu Wei. 2020 · 2012
Earlier work this paper cites.
Unks everywhere: Adapting multilingual language models to new scripts
Jonas Pfeiffer, Ivan Vulic, Iryna Gurevych, and Sebastian Ruder. 2020c · 2012
Earlier work this paper cites.
Japanese and korean voice search
M. Schuster and Kaisuke Nakajima. 2012 · 2012
Earlier work this paper cites.
Orthogonal language and task adapters in zero-shot cross-lingual transfer
Marko Vidoni, Ivan Vulic, and Goran Glavas. 2020 · 2012
Earlier work this paper cites.
A simple, fast, and effective reparameterization of IBM model 2
Chris Dyer, Victor Chahuneau, and Noah A. Smith. 2013 · 2013
Earlier work this paper cites.
Exploiting similarities among languages for machine translation
Tomas Mikolov, Quoc V Le, and Ilya Sutskever. 2013 · 2013
Earlier work this paper cites.
Recursive deep models for semantic compositionality over a sentiment treebank
Richard Socher, Alex Perelygin, Jean Wu, Jason Chuang, Christopher D Manning, Andrew Ng, and Christopher Potts. 2013 · 2013
Earlier work this paper cites.
Neural machine translation by jointly learning to align and translate
Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2016 · 2016
Earlier work this paper cites.
Generating sentences from a continuous space
Samuel R. Bowman, Luke Vilnis, Oriol Vinyals, Andrew Dai, Rafal Jozefowicz, and Samy Bengio. 2016 · 2016
Earlier work this paper cites.
Efficient word alignment with markov chain monte carlo
Robert Östling and Jörg Tiedemann. 2016 · 2016
Earlier work this paper cites.
Google’s neural machine translation system: Bridging the gap between human and machine translation
Yonghui Wu, Mike Schuster, Zhifeng Chen, Quoc V. Le, Mohammad Norouzi, Wolfgang Macherey, Maxim Krikun, Yuan Cao, Qin Gao, Klaus Macherey, Jeff Klingner, Apurva Shah, Melvin Johnson, Xiaobing Liu, Lukasz Kaiser, Stephan Gouws, Yoshikiyo Kato, Taku Kudo, Hideto Kazawa, Keith Stevens, George Kurian, Nishant Patil, Wei Wang, Cliff Young, Jason Smith, Jason Riesa, Alex Rudnick, Oriol Vinyals, Greg Corrado, Macduff Hughes, and Jeffrey Dean. 2016 · 2016
Earlier work this paper cites.
Semeval-2017 task 1: Semantic textual similarity-multilingual and cross-lingual focused evaluation
Daniel Cer, Mona Diab, Eneko Agirre, Inigo Lopez-Gazpio, and Lucia Specia. 2017 · 2017
Earlier work this paper cites.
First quora dataset release: Question pairs
Shankar Iyer, Nikhil Dandekar, and Kornel Csernai. 2017 · 2017
Earlier work this paper cites.
Lightgbm: A highly efficient gradient boosting decision tree
Guolin Ke, Qi Meng, Thomas Finley, Taifeng Wang, Wei Chen, Weidong Ma, Qiwei Ye, and Tie-Yan Liu. 2017 · 2017
Earlier work this paper cites.
Transfer learning across low-resource, related languages for neural machine translation
Toan Q. Nguyen and David Chiang. 2017 · 2017
Earlier work this paper cites.
Learning multiple visual domains with residual adapters
Sylvestre-Alvise Rebuffi, Hakan Bilen, and Andrea Vedaldi. 2017 · 2017
Cited alongside, same era.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017 · 2017
Cited alongside, same era.
Overview of the second BUCC shared task: Spotting parallel sentences in comparable corpora
Pierre Zweigenbaum, Serge Sharoff, and Reinhard Rapp. 2017 · 2017
Cited alongside, same era.
Xnli: Evaluating cross-lingual sentence representations
Alexis Conneau, Ruty Rinott, Guillaume Lample, Adina Williams, Samuel R. Bowman, Holger Schwenk, and Veselin Stoyanov. 2018 · 2018
Cited alongside, same era.
Bert: Pre-training of deep bidirectional transformers for language understanding
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018 · 2018
Contact relatedness can help improve multilingual NMT: Microsoft STCI-MT @ WMT20
Vikrant Goyal, Anoop Kunchukuttan, Rahul Kejriwal, Siddharth Jain, and Amit Bhagwat. 2020 · 2020
Later among the works it cites.
Momentum contrast for unsupervised visual representation learning
Kaiming He, Haoqi Fan, Yuxin Wu, Saining Xie, and Ross Girshick. 2020 · 2020
Later among the works it cites.
XTREME: A massively multilingual multi-task benchmark for evaluating cross-lingual generalisation
Junjie Hu, Sebastian Ruder, Aditya Siddhant, Graham Neubig, Orhan Firat, and Melvin Johnson. 2020 · 2020
Later among the works it cites.
SimAlign: High quality word alignments without parallel training data using static and contextualized embeddings
Masoud Jalili Sabet, Philipp Dufter, François Yvon, and Hinrich Schütze. 2020 · 2020
Later among the works it cites.
Cross-lingual ability of multilingual BERT: an empirical study
Karthikeyan K, Zihan Wang, Stephen Mayhew, and Dan Roth. 2020 · 2020
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Cited alongside, same era.
Sentencepiece: A simple and language independent subword tokenizer and detokenizer for neural text processing
Taku Kudo and John Richardson. 2018a · 2018
Cited alongside, same era.
SentencePiece: A simple and language independent subword tokenizer and detokenizer for neural text processing
Taku Kudo and John Richardson. 2018b · 2018
Cited alongside, same era.
A corpus for multilingual document classification in eight languages
Holger Schwenk and Xian Li. 2018 · 2018
Cited alongside, same era.
GLUE: A multi-task benchmark and analysis platform for natural language understanding
Alex Wang, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel Bowman. 2018a · 2018
Cited alongside, same era.
GLUE: A multi-task benchmark and analysis platform for natural language understanding
Alex Wang, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel R. Bowman. 2018b · 2018
Cited alongside, same era.
A broad-coverage challenge corpus for sentence understanding through inference
Adina Williams, Nikita Nangia, and Samuel Bowman. 2018 · 2018
Cited alongside, same era.
Massively multilingual sentence embeddings for zero-shot cross-lingual transfer and beyond
Mikel Artetxe and Holger Schwenk. 2019 · 2019
Cited alongside, same era.
Later among the works it cites.
IndicNLPSuite: Monolingual Corpora, Evaluation Benchmarks and Pre-trained Multilingual Language Models for Indian Languages
Divyanshu Kakwani, Anoop Kunchukuttan, Satish Golla, Gokul N.C., Avik Bhattacharyya, Mitesh M. Khapra, and Pratyush Kumar. 2020 · 2020
Later among the works it cites.
Flaubert: Unsupervised language model pre-training for french
Hang Le, Loïc Vial, Jibril Frej, Vincent Segonne, Maximin Coavoux, Benjamin Lecouteux, Alexandre Allauzen, Benoît Crabbé, Laurent Besacier, and Didier Schwab. 2020 · 2020
Later among the works it cites.
MLQA: evaluating cross-lingual extractive question answering
Patrick S. H. Lewis, Barlas Oguz, Ruty Rinott, Sebastian Riedel, and Holger Schwenk. 2020 · 2020
Later among the works it cites.
XGLUE: A new benchmark datasetfor cross-lingual pre-training, understanding and generation
Yaobo Liang, Nan Duan, Yeyun Gong, Ning Wu, Fenfei Guo, Weizhen Qi, Ming Gong, Linjun Shou, Daxin Jiang, Guihong Cao, Xiaodong Fan, Ruofei Zhang, Rahul Agrawal, Edward Cui, Sining Wei, Taroon Bharti, Ying Qiao, Jiun-Hung Chen, Winnie Wu, Shuguang Liu, Fan Yang, Daniel Campos, Rangan Majumder, and Ming Zhou. 2020 · 2020
Later among the works it cites.
On the language neutrality of pre-trained multilingual representations
Jindřich Libovický, Rudolf Rosa, and Alexander Fraser. 2020 · 2020
Later among the works it cites.
Universal dependencies according to BERT: both more specific and more general
Tomasz Limisiewicz, David Marecek, and Rudolf Rosa. 2020 · 2020
Later among the works it cites.
CamemBERT: a tasty French language model
Louis Martin, Benjamin Muller, Pedro Javier Ortiz Suárez, Yoann Dupont, Laurent Romary, Éric de la Clergerie, Djamé Seddah, and Benoît Sagot. 2020 · 2020
Later among the works it cites.
Benjamin Muller, Benoit Sagot, and Djamé Seddah. 2020 · 2020
Later among the works it cites.
MAD-X: an adapter-based framework for multi-task cross-lingual transfer
Jonas Pfeiffer, Ivan Vulic, Iryna Gurevych, and Sebastian Ruder. 2020b · 2020
Later among the works it cites.
English intermediate-task training improves zero-shot cross-lingual transfer too
Jason Phang, Iacer Calixto, Phu Mon Htut, Yada Pruksachatkun, Haokun Liu, Clara Vania, Katharina Kann, and Samuel R. Bowman. 2020 · 2020
Later among the works it cites.
XCOPA: A multilingual dataset for causal commonsense reasoning
Edoardo M. Ponti, Goran Glavaš, Olga Majewska, Qianchu Liu, Ivan Vulić, and Anna Korhonen. 2020 · 2020
Later among the works it cites.
Adaptation of deep bidirectional transformers for Afrikaans language
Sello Ralethe. 2020 · 2020
Later among the works it cites.
Multi2oie: Multilingual open information extraction based on multi-head attention with bert
Youngbin Ro, Yukyung Lee, and Pilsung Kang. 2020 · 2020
Later among the works it cites.
A primer in bertology: What we know about how bert works
Anna Rogers, Olga Kovaleva, and Anna Rumshisky. 2020 · 2020
Later among the works it cites.
LAReQA: Language-agnostic answer retrieval from a multilingual pool
Uma Roy, Noah Constant, Rami Al-Rfou, Aditya Barua, Aaron Phillips, and Yinfei Yang. 2020 · 2020
Later among the works it cites.
Galileo at SemEval-2020 task 12: Multi-lingual learning for offensive language identification using pre-trained language models
Shuohuan Wang, Jiaxiang Liu, Xuan Ouyang, and Yu Sun. 2020a · 2020
Later among the works it cites.
Structure-level knowledge distillation for multilingual sequence labeling
Xinyu Wang, Yong Jiang, Nguyen Bach, Tao Wang, Fei Huang, and Kewei Tu. 2020b · 2020
Later among the works it cites.
Extending multilingual BERT to low-resource languages
Zihan Wang, Karthikeyan K, Stephen Mayhew, and Dan Roth. 2020c · 2020
Later among the works it cites.
Cross-lingual alignment vs joint training: A comparative study and A simple unified framework
Zirui Wang, Jiateng Xie, Ruochen Xu, Yiming Yang, Graham Neubig, and Jaime G. Carbonell. 2020d · 2020
Later among the works it cites.
Are all languages created equal in multilingual bert?
Shijie Wu and Mark Dredze. 2020 · 2020
Later among the works it cites.
Alternating language modeling for cross-lingual pre-training
Jian Yang, Shuming Ma, Dongdong Zhang, ShuangZhi Wu, Zhoujun Li, and Ming Zhou. 2020 · 2020
Later among the works it cites.
Semeval-2020 task 12: Multilingual offensive language identification in social media (offenseval 2020)
Marcos Zampieri, Preslav Nakov, Sara Rosenthal, Pepa Atanasova, Georgi Karadzhov, Hamdy Mubarak, Leon Derczynski, Zeses Pitenis, and Çagri Çöltekin. 2020 · 2020
Later among the works it cites.
End-to-end neural word alignment outperforms GIZA++
Thomas Zenkel, Joern Wuebker, and John DeNero. 2020 · 2020
Later among the works it cites.
InfoXLM: An information-theoretic framework for cross-lingual language model pre-training
Zewen Chi, Li Dong, Furu Wei, Nan Yang, Saksham Singhal, Wenhui Wang, Xia Song, Xian-Ling Mao, Heyan Huang, and Ming Zhou. 2021a · 2021
Closest in time.
Improving pretrained cross-lingual language models via self-labeled word alignment
Zewen Chi, Li Dong, Bo Zheng, Shaohan Huang, Xian-Ling Mao, Heyan Huang, and Furu Wei. 2021b · 2021
Closest in time.
Rethinking embedding coupling in pre-trained language models
Hyung Won Chung, Thibault Févry, Henry Tsai, Melvin Johnson, and Sebastian Ruder. 2021b · 2021
Closest in time.
Word alignment by fine-tuning embeddings on parallel corpora
Zi-Yi Dou and Graham Neubig. 2021 · 2021
Closest in time.
Explicit alignment objectives for multilingual bidirectional encoders
Junjie Hu, Melvin Johnson, Orhan Firat, Aditya Siddhant, and Graham Neubig. 2021 · 2021
Closest in time.
Ammus : A survey of transformer-based pretrained models in natural language processing
Katikapalli Subramanyam Kalyan, Ajit Rajasekharan, and S. Sangeetha. 2021 · 2021
Closest in time.
Explainaboard: An explainable leaderboard for nlp
Pengfei Liu, Jinlan Fu, Yang Xiao, Weizhe Yuan, Shuaicheng Chang, Junqi Dai, Yixin Liu, Zihuiwen Ye, and Graham Neubig. 2021 · 2021
Closest in time.
{VECO}: Variable encoder-decoder pre-training for cross-lingual understanding and generation
Fuli Luo, Wei Wang, Jiahao Liu, Yijia Liu, Bin Bi, Songfang Huang, Fei Huang, and Luo Si. 2021 · 2021
Closest in time.
Trankit: A light-weight transformer-based toolkit for multilingual natural language processing
Minh Nguyen, Viet Lai, Amir Pouran Ben Veyseh, and Thien Huu Nguyen. 2021 · 2021
Closest in time.
Xuan Ouyang, Shuohuan Wang, Chao Pang, Yu Sun, Hao Tian, Hua Wu, and Haifeng Wang. 2021 · 2021
Closest in time.
XTREME-R: towards more challenging and nuanced multilingual evaluation
Sebastian Ruder, Noah Constant, Jan Botha, Aditya Siddhant, Orhan Firat, Jinlan Fu, Pengfei Liu, Junjie Hu, Graham Neubig, and Melvin Johnson. 2021 · 2021
Closest in time.
How good is your tokenizer? on the monolingual performance of multilingual language models
Phillip Rust, Jonas Pfeiffer, Ivan Vulic, Sebastian Ruder, and Iryna Gurevych. 2021 · 2021
Closest in time.
Revisiting the primacy of english in zero-shot cross-lingual transfer
Iulia Turc, Kenton Lee, Jacob Eisenstein, Ming-Wei Chang, and Kristina Toutanova. 2021 · 2021
Closest in time.
On learning universal representations across languages
Xiangpeng Wei, Rongxiang Weng, Yue Hu, Luxi Xing, Heng Yu, and Weihua Luo. 2021 · 2021
Closest in time.
mt5: A massively multilingual pre-trained text-to-text transformer
Linting Xue, Noah Constant, Adam Roberts, Mihir Kale, Rami Al-Rfou, Aditya Siddhant, Aditya Barua, and Colin Raffel. 2021 · 2021
Closest in time.