Fetching the paper…
Reading the bibliography…
Knowledge-Enhanced Pre-trained Language Models (KEPLMs) are pre-trained models with relation triples injecting from knowledge graphs to improve language understanding abilities.
Transformer-xl: Attentive language models beyond a fixed-length context
Dai, Z.; Yang, Z.; Yang, Y.; Carbonell, J.; Le, Q. V.; and Salakhutdinov, R. 2019 · 1901
Earlier work this paper cites.
ERNIE: Enhanced Representation through Knowledge Integration
Sun, Y.; Wang, S.; Li, Y.; Feng, S.; Chen, X.; Zhang, H.; Tian, X.; Zhu, D.; Tian, H.; and Wu, H. 2019 · 1904
Earlier work this paper cites.
BERT is Not a Knowledge Base (Yet): Factual Knowledge vs. Name-Based Reasoning in Unsupervised QA
Pörner, N.; Waltinger, U.; and Schütze, H. 2019 · 1911
Earlier work this paper cites.
KEPLER: A Unified Model for Knowledge Embedding and Pre-trained Language Representation
Wang, X.; Gao, T.; Zhu, Z.; Liu, Z.; Li, J.; and Tang, J. 2019b · 1911
Earlier work this paper cites.
Contextual knowledge selection and embedding towards enhanced pre-trained language models
Su, Y.; Han, X.; Zhang, Z.; Li, P.; Liu, Z.; Lin, Y.; Zhou, J.; and Sun, M. 2020 · 2009
Earlier work this paper cites.
TAGME: on-the-fly annotation of short text fragments (by wikipedia entities)
Ferragina, P.; and Scaiella, U. 2010 · 2010
Earlier work this paper cites.
Language Models are Open Knowledge Graphs
Wang, C.; Liu, X.; and Song, D. 2020 · 2010
Earlier work this paper cites.
Translating Embeddings for Modeling Multi-relational Data
Bordes, A.; Usunier, N.; García-Durán, A.; Weston, J.; and Yakhnenko, O. 2013 · 2013
Earlier work this paper cites.
On Using Very Large Target Vocabulary for Neural Machine Translation
Jean, S.; Cho, K.; Memisevic, R.; and Bengio, Y. 2015 · 2015
Earlier work this paper cites.
Ba, L. J.; Kiros, J. R.; and Hinton, G. E. 2016 · 2016
Earlier work this paper cites.
A Structured Self-Attentive Sentence Embedding
Lin, Z.; Feng, M.; dos Santos, C. N.; Yu, M.; Xiang, B.; Zhou, B.; and Bengio, Y. 2017 · 2017
Earlier work this paper cites.
Position-aware Attention and Supervised Data Improve Slot Filling
Zhang, Y.; Zhong, V.; Chen, D.; Angeli, G.; and Manning, C. D. 2017 · 2017
Earlier work this paper cites.
Ultra-Fine Entity Typing
Choi, E.; Levy, O.; Choi, Y.; and Zettlemoyer, L. 2018 · 2018
Earlier work this paper cites.
Graph Convolution over Pruned Dependency Trees Improves Relation Extraction
Zhang, Y.; and Qi, P. 2018 · 2018
Cited alongside, same era.
Investigating Entity Knowledge in BERT with Simple Neural End-To-End Entity Linking
Broscheit, S. 2019 · 2019
Cited alongside, same era.
Fine-tune BERT with Sparse Self-Attention Mechanism
Cui, B.; Li, Y.; Chen, M.; and Zhang, Z. 2019 · 2019
Cited alongside, same era.
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J.; Chang, M.; Lee, K.; and Toutanova, K. 2019 · 2019
Cited alongside, same era.
Multi-Task Deep Neural Networks for Natural Language Understanding
Liu, X.; He, P.; Chen, W.; and Gao, J. 2019 · 2019
Cited alongside, same era.
Knowledge Enhanced Contextual Word Representations
Peters, M. E.; Neumann, M.; IV, R. L. L.; Schwartz, R.; Joshi, V.; Singh, S.; and Smith, N. A. 2019 · 2019
StructBERT: Incorporating Language Structures into Pre-training for Deep Language Understanding
Wang, W.; Bi, B.; Yan, M.; Wu, C.; Xia, J.; Bao, Z.; Peng, L.; and Si, L. 2020 · 2020
Later among the works it cites.
Summarizing Chinese Medical Answer with Graph Convolution Networks and Question-focused Dual Attention
Zhang, N.; Deng, S.; Li, J.; Chen, X.; Zhang, W.; and Chen, H. 2020 · 2020
Later among the works it cites.
Knowledgeable or Educated Guess? Revisiting Language Models as Knowledge Bases
Cao, B.; Lin, H.; Han, X.; Sun, L.; Yan, L.; Liao, M.; Xue, T.; and Xu, J. 2021 · 2021
Closest in time.
Convolutions and Self-Attention: Re-interpreting Relative Positions in Pre-trained Language Models
Chang, T. A.; Xu, Y.; Xu, W.; and Tu, Z. 2021 · 2021
Closest in time.
Combining pre-trained language models and structured knowledge
Colon-Hernandez, P.; Havasi, C.; Alonso, J. B.; Huggins, M.; and Breazeal, C. 2021 · 2021
Closest in time.
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Cited alongside, same era.
Language Models as Knowledge Bases?
Petroni, F.; Rocktäschel, T.; Riedel, S.; Lewis, P. S. H.; Bakhtin, A.; Wu, Y.; and Miller, A. H. 2019 · 2019
Cited alongside, same era.
XLNet: Generalized Autoregressive Pretraining for Language Understanding
Yang, Z.; Dai, Z.; Yang, Y.; Carbonell, J. G.; Salakhutdinov, R.; and Le, Q. V. 2019 · 2019
Cited alongside, same era.
ERNIE: Enhanced Language Representation with Informative Entities
Zhang, Z.; Han, X.; Liu, Z.; Jiang, X.; Sun, M.; and Liu, Q. 2019 · 2019
Cited alongside, same era.
Infusing Disease Knowledge into BERT for Health Question Answering, Medical Inference and Disease Name Recognition
He, Y.; Zhu, Z.; Zhang, Y.; Chen, Q.; and Caverlee, J. 2020 · 2020
Cited alongside, same era.
SpanBERT: Improving Pre-training by Representing and Predicting Spans
Joshi, M.; Chen, D.; Liu, Y.; Weld, D. S.; Zettlemoyer, L.; and Levy, O. 2020 · 2020
Cited alongside, same era.
K-BERT: Enabling Language Representation with Knowledge Graph
Liu, W.; Zhou, P.; Zhao, Z.; Wang, Z.; Ju, Q.; Deng, H.; and Wang, P. 2020 · 2020
Cited alongside, same era.
On Commonsense Cues in BERT for Solving Commonsense Tasks
Cui, L.; Cheng, S.; Wu, Y.; and Zhang, Y. 2021 · 2021
Closest in time.
Named Entity Recognition with Small Strongly Labeled and Large Weakly Labeled Data
Jiang, H.; Zhang, D.; Cao, T.; Yin, B.; and Zhao, T. 2021 · 2021
Closest in time.
ILDC for CJPE: Indian Legal Documents Corpus for Court Judgment Prediction and Explanation
Malik, V.; Sanjay, R.; Nigam, S. K.; Ghosh, K.; Guha, S. K.; Bhattacharya, A.; and Modi, A. 2021 · 2021
Closest in time.
Locate and Label: A Two-stage Identifier for Nested Named Entity Recognition
Shen, Y.; Ma, X.; Tan, Z.; Zhang, S.; Wang, W.; and Lu, W. 2021 · 2021
Closest in time.
K-Adapter: Infusing Knowledge into Pre-Trained Models with Adapters
Wang, R.; Tang, D.; Duan, N.; Wei, Z.; Huang, X.; Ji, J.; Cao, G.; Jiang, D.; and Zhou, M. 2021 · 2021
Closest in time.
Syntax-Enhanced Pre-trained Model
Xu, Z.; Guo, D.; Tang, D.; Su, Q.; Shou, L.; Gong, M.; Zhong, W.; Quan, X.; Jiang, D.; and Duan, N. 2021 · 2021
Closest in time.
Drop Redundant, Shrink Irrelevant: Selective Knowledge Injection for Language Pretraining
Zhang, N.; Deng, S.; Cheng, X.; Chen, X.; Zhang, Y.; Zhang, W.; Chen, H.; and Center, H. I. 2021 · 2021
Closest in time.