Fetching the paper…
Reading the bibliography…
Incorporating factual knowledge into pre-trained language models (PLM) such as BERT is an emerging trend in recent NLP studies.
Wordnet: A lexical database for english
George A. Miller · 1995
Earlier work this paper cites.
Nltk: The natural language toolkit
Edward Loper and Steven Bird · 2002
Earlier work this paper cites.
Toward an architecture for never-ending language learning
A. Carlson, J. Betteridge, B. Kisiel, B. Settles, E.R. Hruschka Jr., and T.M. Mitchell · 2010
Earlier work this paper cites.
Tagme: on-the-fly annotation of short text fragments (by wikipedia entities)
P. Ferragina and Ugo Scaiella · 2010
Earlier work this paper cites.
Choice of plausible alternatives: An evaluation of commonsense causal reasoning
Melissa Roemmele, Cosmin Adrian Bejan, and Andrew S. Gordon · 2011
Earlier work this paper cites.
Translating embeddings for modeling multi-relational data
Antoine Bordes, Nicolas Usunier, Alberto García-Durán, J. Weston, and Oksana Yakhnenko · 2013
Earlier work this paper cites.
Embedding entities and relations for learning and inference in knowledge bases, 2015
Bishan Yang, Wen tau Yih, Xiaodong He, Jianfeng Gao, and Li Deng · 2015
Earlier work this paper cites.
Squad: 100,000+ questions for machine comprehension of text, 2016
Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, and Percy Liang · 2016
Earlier work this paper cites.
Semi-supervised classification with graph convolutional networks, 2017
Thomas N. Kipf and Max Welling · 2017
Earlier work this paper cites.
Modeling relational data with graph convolutional networks, 2017
Michael Schlichtkrull, Thomas N. Kipf, Peter Bloem, Rianne van den Berg, Ivan Titov, and Max Welling · 2017
Earlier work this paper cites.
Attention is all you need, 2017
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin · 2017
Earlier work this paper cites.
Leveraging knowledge bases in LSTMs for improving machine reading
Bishan Yang and Tom Mitchell · 2017
Cited alongside, same era.
Looking beyond the surface:a challenge set for reading comprehension over multiple sentences
Daniel Khashabi, Snigdha Chaturvedi, Michael Roth, Shyam Upadhyay, and Dan Roth · 2018
Cited alongside, same era.
Graph attention networks, 2018
Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Liò, and Yoshua Bengio · 2018
Cited alongside, same era.
Record: Bridging the gap between human and machine commonsense reading comprehension, 2018
Sheng Zhang, Xiaodong Liu, Jingjing Liu, Jianfeng Gao, Kevin Duh, and Benjamin Van Durme · 2018
Cited alongside, same era.
BERT: Pre-training of deep bidirectional transformers for language understanding
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova · 2019
Cited alongside, same era.
Graph star net for generalized multi-task learning, 2019
Enhancing pre-trained language representations with rich knowledge for machine reading comprehension
An Yang, Quan Wang, Jing Liu, Kai Liu, Yajuan Lyu, Hua Wu, Qiaoqiao She, and Sujian Li · 2019
Later among the works it cites.
Ernie: Enhanced language representation with informative entities, 2019
Zhengyan Zhang, Xu Han, Zhiyuan Liu, Xin Jiang, Maosong Sun, and Qun Liu · 2019
Later among the works it cites.
Dense: An enhanced non-abelian group representation for knowledge graph embedding, 2020
Haonan Lu and Hailin Hu · 2020
Later among the works it cites.
Exploring the limits of transfer learning with a unified text-to-text transformer, 2020
Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu · 2020
Later among the works it cites.
Cokebert: Contextual knowledge selection and embedding towards enhanced pre-trained language models, 2020
Yusheng Su, Xu Han, Zhengyan Zhang, Peng Li, Zhiyuan Liu, Yankai Lin, Jie Zhou, and Maosong Sun · 2020
Later among the works it cites.
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Lu Haonan, Seth H. Huang, Tian Ye, and Guo Xiuyan · 2019
Cited alongside, same era.
Knowledge enhanced contextual word representations, 2019
Matthew E. Peters, Mark Neumann, Robert L. Logan IV au2, Roy Schwartz, Vidur Joshi, Sameer Singh, and Noah A. Smith · 2019
Cited alongside, same era.
Bert is not a knowledge base (yet): Factual knowledge vs. name-based reasoning in unsupervised qa
Nina Poerner, Ulli Waltinger, and Hinrich Schütze · 2019
Cited alongside, same era.
Machine reading comprehension using structural knowledge graph-aware network
Delai Qiu, Yuanzhe Zhang, Xinwei Feng, Xiangwen Liao, Wenbin Jiang, Yajuan Lyu, Kang Liu, and Jun Zhao · 2019
Cited alongside, same era.
Pretrained encyclopedia: Weakly supervised knowledge-pretrained language model, 2019
Wenhan Xiong, Jingfei Du, William Yang Wang, and Veselin Stoyanov · 2019
Cited alongside, same era.
K-bert: Enabling language representation with knowledge graph, 2019a
Weijie Liu, Peng Zhou, Zhe Zhao, Zhiruo Wang, Qi Ju, Haotang Deng, and Ping Wang
Cited in the paper.
Roberta: A robustly optimized bert pretraining approach, 2019b
Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov
Cited in the paper.
Colake: Contextualized language and knowledge embedding, 2020
Tianxiang Sun, Yunfan Shao, Xipeng Qiu, Qipeng Guo, Yaru Hu, Xuanjing Huang, and Zheng Zhang · 2020
Later among the works it cites.
Huggingface’s transformers: State-of-the-art natural language processing, 2020
Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mariama Drame, Quentin Lhoest, and Alexander M. Rush · 2020
Later among the works it cites.
Luke: Deep contextualized entity representations with entity-aware self-attention, 2020
Ikuya Yamada, Akari Asai, Hiroyuki Shindo, Hideaki Takeda, and Yuji Matsumoto · 2020
Later among the works it cites.
Xlnet: Generalized autoregressive pretraining for language understanding, 2020
Zhilin Yang, Zihang Dai, Yiming Yang, Jaime Carbonell, Ruslan Salakhutdinov, and Quoc V. Le · 2020
Later among the works it cites.
Jaket: Joint pre-training of knowledge graph and language understanding, 2020
Donghan Yu, Chenguang Zhu, Yiming Yang, and Michael Zeng · 2020
Later among the works it cites.