Fetching the paper…
Reading the bibliography…
Cross-lingual pre-training has achieved great successes using monolingual and bilingual plain text corpora.
Roberta: A robustly optimized bert pretraining approach
Liu, Y.; Ott, M.; Goyal, N.; Du, J.; Joshi, M.; Chen, D.; Levy, O.; Lewis, M.; Zettlemoyer, L.; and Stoyanov, V. 2019 · 1907
Earlier work this paper cites.
Knowledge-Aware Language Model Pretraining
Rosset, C.; Xiong, C.; Phan, M.; Song, X.; Bennett, P.; and Tiwary, S. 2020 · 2007
Earlier work this paper cites.
Representing General Relational Knowledge in ConceptNet 5
Speer, R.; and Havasi, C. 2012 · 2012
Earlier work this paper cites.
Cross-Lingual Semantic Similarity of Words as the Similarity of Their Semantic Word Responses
Vulić, I.; and Moens, M.-F. 2013 · 2013
Earlier work this paper cites.
Wikidata: a free collaborative knowledgebase
Vrandečić, D.; and Krötzsch, M. 2014 · 2014
Earlier work this paper cites.
Knowledge graph and text jointly embedding
Wang, Z.; Zhang, J.; Feng, J.; and Chen, Z. 2014 · 2014
Earlier work this paper cites.
Adam: A method for stochastic optimization
Kingma, D. P.; and Ba, J. 2015 · 2015
Earlier work this paper cites.
Representing text for joint embedding of text and knowledge bases
Toutanova, K.; Chen, D.; Pantel, P.; Poon, H.; Choudhury, P.; and Gamon, M. 2015 · 2015
Earlier work this paper cites.
Joint representation learning of text and knowledge for knowledge graph completion
Han, X.; Liu, Z.; and Sun, M. 2016 · 2016
Earlier work this paper cites.
SQuAD: 100,000+ Questions for Machine Comprehension of Text
Rajpurkar, P.; Zhang, J.; Lopyrev, K.; and Liang, P. 2016 · 2016
Earlier work this paper cites.
Text-enhanced representation learning for knowledge graph
Wang, Z.; and Li, J.-Z. 2016 · 2016
Earlier work this paper cites.
Knowledge representation via joint learning of sequential text and knowledge graphs
Wu, J.; Xie, R.; Liu, Z.; and Sun, M. 2016 · 2016
Earlier work this paper cites.
Joint Learning of the Embedding of Words and Entities for Named Entity Disambiguation
Yamada, I.; Shindo, H.; Takeda, H.; and Takefuji, Y. 2016 · 2016
Earlier work this paper cites.
Bridge text and knowledge by learning multi-prototype entity mention embedding
Cao, Y.; Huang, L.; Ji, H.; Chen, X.; and Li, J. 2017 · 2017
Earlier work this paper cites.
Conceptnet 5.5: An open multilingual graph of general knowledge
Speer, R.; Chin, J.; and Havasi, C. 2017 · 2017
Earlier work this paper cites.
Attention is all you need
Vaswani, A.; Shazeer, N.; Parmar, N.; Uszkoreit, J.; Jones, L.; Gomez, A. N.; Kaiser, Ł.; and Polosukhin, I. 2017 · 2017
Earlier work this paper cites.
XNLI: Evaluating Cross-lingual Sentence Representations
Conneau, A.; Rinott, R.; Lample, G.; Williams, A.; Bowman, S.; Schwenk, H.; and Stoyanov, V. 2018 · 2018
Cited alongside, same era.
T-REx: A Large Scale Alignment of Natural Language with Knowledge Base Triples
Elsahar, H.; Vougiouklis, P.; Remaci, A.; Gravier, C.; Hare, J.; Laforest, F.; and Simperl, E. 2018 · 2018
Cited alongside, same era.
Investigating Entity Knowledge in BERT with Simple Neural End-To-End Entity Linking
Broscheit, S. 2019 · 2019
Cited alongside, same era.
Cross-lingual language model pretraining
Conneau, A.; and Lample, G. 2019 · 2019
Cited alongside, same era.
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J.; Chang, M.-W.; Lee, K.; and Toutanova, K. 2019 · 2019
Cited alongside, same era.
Unicoder: A Universal Language Encoder by Pre-training with Multiple Cross-lingual Tasks
MLQA: Evaluating Cross-lingual Extractive Question Answering
Lewis, P.; Oguz, B.; Rinott, R.; Riedel, S.; and Schwenk, H. 2020 · 2020
Later among the works it cites.
Xglue: A new benchmark dataset for cross-lingual pre-training, understanding and generation
Liang, Y.; Duan, N.; Gong, Y.; Wu, N.; Guo, F.; Qi, W.; Gong, M.; Shou, L.; Jiang, D.; Cao, G.; Fan, X.; Zhang, R.; Agrawal, R.; Cui, E.; Wei, S.; Bharti, T.; Qiao, Y.; Chen, J.-H.; Wu, W.; Liu, S.; Yang, F.; Campos, D.; Majumder, R.; and Zhou, M. 2020 · 2020
Later among the works it cites.
K-bert: Enabling language representation with knowledge graph
Liu, W.; Zhou, P.; Zhao, Z.; Wang, Z.; Ju, Q.; Deng, H.; and Wang, P. 2020 · 2020
Later among the works it cites.
Compositional and Lexical Semantics in RoBERTa, BERT and DistilBERT: A Case Study on CoQA
Staliūnaitė, I.; and Iacobacci, I. 2020 · 2020
Later among the works it cites.
Ccnet: Extracting high quality monolingual datasets from web crawl data
Wenzek, G.; Lachaux, M.-A.; Conneau, A.; Chaudhary, V.; Guzman, F.; Joulin, A.; and Grave, E. 2020 · 2020
Later among the works it cites.
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Huang, H.; Liang, Y.; Duan, N.; Gong, M.; Shou, L.; Jiang, D.; and Zhou, M. 2019 · 2019
Cited alongside, same era.
Cross-lingual Joint Entity and Word Embedding to Improve Entity Linking and Parallel Sentence Mining
Pan, X.; Gowda, T.; Ji, H.; May, J.; and Miller, S. 2019 · 2019
Cited alongside, same era.
Language Models as Knowledge Bases?
Petroni, F.; Rocktäschel, T.; Lewis, P.; Bakhtin, A.; Wu, Y.; Miller, A. H.; and Riedel, S. 2019 · 2019
Cited alongside, same era.
A survey of cross-lingual word embedding models
Ruder, S.; Vulić, I.; and Søgaard, A. 2019 · 2019
Cited alongside, same era.
ERNIE: Enhanced Language Representation with Informative Entities
Zhang, Z.; Han, X.; Liu, Z.; Jiang, X.; Sun, M.; and Liu, Q. 2019 · 2019
Cited alongside, same era.
A simple framework for contrastive learning of visual representations
Chen, T.; Kornblith, S.; Norouzi, M.; and Hinton, G. 2020 · 2020
Cited alongside, same era.
Unsupervised cross-lingual representation learning at scale
Conneau, A.; Khandelwal, K.; Goyal, N.; Chaudhary, V.; Wenzek, G.; Guzmán, F.; Grave, E.; Ott, M.; Zettlemoyer, L.; and Stoyanov, V. 2020 · 2020
Cited alongside, same era.
Pretrained encyclopedia: Weakly supervised knowledge-pretrained language model
Xiong, W.; Du, J.; Wang, W. Y.; and Stoyanov, V. 2020 · 2020
Later among the works it cites.
Alternating Language Modeling for Cross-Lingual Pre-Training
Yang, J.; Ma, S.; Zhang, D.; Wu, S.; Li, Z.; and Zhou, M. 2020 · 2020
Later among the works it cites.
Cross-lingual Entity Alignment with Incidental Supervision
Chen, M.; Shi, W.; Zhou, B.; and Roth, D. 2021 · 2021
Closest in time.
Infoxlm: An information-theoretic framework for cross-lingual language model pre-training
Chi, Z.; Dong, L.; Wei, F.; Yang, N.; Singhal, S.; Wang, W.; Song, X.; Mao, X.-L.; Huang, H.; and Zhou, M. 2021 · 2021
Closest in time.
Discovering Representation Sprachbund For Multilingual Pre-Training
Fan, Y.; Liang, Y.; Muzio, A.; Hassan, H.; Li, H.; Zhou, M.; and Duan, N. 2021 · 2021
Closest in time.
Explicit Alignment Objectives for Multilingual Bidirectional Encoders
Hu, J.; Johnson, M.; Firat, O.; Siddhant, A.; and Neubig, G. 2021 · 2021
Closest in time.
Multilingual LAMA: Investigating Knowledge in Multilingual Pretrained Language Models
Kassner, N.; Dufter, P.; and Schütze, H. 2021 · 2021
Closest in time.
ERNIE-M: Enhanced Multilingual Representation by Aligning Cross-lingual Semantics with Monolingual Corpora
Ouyang, X.; Wang, S.; Pang, C.; Sun, Y.; Tian, H.; Wu, H.; and Wang, H. 2021 · 2021
Closest in time.
Multilingual BERT Post-Pretraining Alignment
Pan, L.; Hang, C.-W.; Qi, H.; Shah, A.; Yu, M.; and Potdar, S. 2021 · 2021
Closest in time.
Grounded Conversation Generation as Guided Traverses in Commonsense Knowledge Graphs
Zhang, H.; Liu, Z.; Xiong, C.; and Liu, Z. 2020 · 2043
Closest in time.