Fetching the paper…
Reading the bibliography…
Injecting external knowledge can improve the performance of pre-trained language models (PLMs) on various downstream NLP tasks.
Roberta: A robustly optimized BERT pretraining approach
Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019 · 1907
Earlier work this paper cites.
REALM: retrieval-augmented language model pre-training
Kelvin Guu, Kenton Lee, Zora Tung, Panupong Pasupat, and Ming-Wei Chang. 2020 · 2002
Earlier work this paper cites.
Improving neural networks by preventing co-adaptation of feature detectors
Geoffrey E Hinton, Nitish Srivastava, Alex Krizhevsky, Ilya Sutskever, and Ruslan R Salakhutdinov. 2012 · 2012
Earlier work this paper cites.
Translating embeddings for modeling multi-relational data
Antoine Bordes, Nicolas Usunier, Alberto García-Durán, Jason Weston, and Oksana Yakhnenko. 2013 · 2013
Earlier work this paper cites.
Adam: A method for stochastic optimization
Diederik P. Kingma and Jimmy Ba. 2015 · 2015
Earlier work this paper cites.
Fewrel: A large-scale supervised few-shot relation classification dataset with state-of-the-art evaluation
Xu Han, Hao Zhu, Pengfei Yu, Ziyun Wang, Yuan Yao, Zhiyuan Liu, and Maosong Sun. 2018 · 2018
Earlier work this paper cites.
Universal language model fine-tuning for text classification
Jeremy Howard and Sebastian Ruder. 2018 · 2018
Earlier work this paper cites.
Improving neural fine-grained entity typing with knowledge attention
Ji Xin, Yankai Lin, Zhiyuan Liu, and Maosong Sun. 2018 · 2018
Earlier work this paper cites.
COMET: commonsense transformers for automatic knowledge graph construction
Antoine Bosselut, Hannah Rashkin, Maarten Sap, Chaitanya Malaviya, Asli Celikyilmaz, and Yejin Choi. 2019 · 2019
Earlier work this paper cites.
BERT: Pre-training of deep bidirectional transformers for language understanding
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019 · 2019
Earlier work this paper cites.
Fewrel 2.0: Towards more challenging few-shot relation classification
Tianyu Gao, Xu Han, Hao Zhu, Zhiyuan Liu, Peng Li, Maosong Sun, and Jie Zhou. 2019 · 2019
Earlier work this paper cites.
OpenNRE: An open and extensible toolkit for neural relation extraction
Xu Han, Tianyu Gao, Yuan Yao, Deming Ye, Zhiyuan Liu, and Maosong Sun. 2019 · 2019
Earlier work this paper cites.
Parameter-efficient transfer learning for NLP
Neil Houlsby, Andrei Giurgiu, Stanislaw Jastrzebski, Bruna Morrone, Quentin de Laroussilhe, Andrea Gesmundo, Mona Attariyan, and Sylvain Gelly. 2019 · 2019
Earlier work this paper cites.
Kagnet: Knowledge-aware graph networks for commonsense reasoning
Bill Yuchen Lin, Xinyue Chen, Jamin Chen, and Xiang Ren. 2019 · 2019
Earlier work this paper cites.
Knowledge enhanced contextual word representations
Matthew E. Peters, Mark Neumann, Robert L. Logan IV, Roy Schwartz, Vidur Joshi, Sameer Singh, and Noah A. Smith. 2019 · 2019
Earlier work this paper cites.
Language models are unsupervised multitask learners
Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. 2019 · 2019
Earlier work this paper cites.
ERNIE: enhanced language representation with informative entities
Zhengyan Zhang, Xu Han, Zhiyuan Liu, Xin Jiang, Maosong Sun, and Qun Liu. 2019 · 2019
Earlier work this paper cites.
GEAR: graph-based evidence aggregating and reasoning for fact verification
Jie Zhou, Xu Han, Cheng Yang, Zhiyuan Liu, Lifeng Wang, Changcheng Li, and Maosong Sun. 2019 · 2019
Cited alongside, same era.
KGPT: knowledge-grounded pre-training for data-to-text generation
Wenhu Chen, Yu Su, Xifeng Yan, and William Yang Wang. 2020 · 2020
Cited alongside, same era.
Plug and play language models: A simple approach to controlled text generation
Sumanth Dathathri, Andrea Madotto, Janice Lan, Jane Hung, Eric Frank, Piero Molino, Jason Yosinski, and Rosanne Liu. 2020 · 2020
Cited alongside, same era.
A knowledge-enhanced pretraining model for commonsense story generation
Jian Guan, Fei Huang, Minlie Huang, Zhihao Zhao, and Xiaoyan Zhu. 2020 · 2020
Cited alongside, same era.
Dense passage retrieval for open-domain question answering
Vladimir Karpukhin, Barlas Oguz, Sewon Min, Patrick S. H. Lewis, Ledell Wu, Sergey Edunov, Danqi Chen, and Wen-tau Yih. 2020 · 2020
Cited alongside, same era.
ERICA: Improving entity and relation understanding for pre-trained language models via contrastive learning
Yujia Qin, Yankai Lin, Ryuichi Takanobu, Zhiyuan Liu, Peng Li, Heng Ji, Minlie Huang, Maosong Sun, and Jie Zhou. 2021 · 2021
Later among the works it cites.
Simple entity-centric questions challenge dense retrievers
Christopher Sciavolino, Zexuan Zhong, Jinhyuk Lee, and Danqi Chen. 2021 · 2021
Later among the works it cites.
CokeBERT: Contextual knowledge selection and embedding towards enhanced pre-trained language models
Yusheng Su, Xu Han, Zhengyan Zhang, Yankai Lin, Peng Li, Zhiyuan Liu, Jie Zhou, and Maosong Sun. 2021 · 2021
Later among the works it cites.
K-PLUG: knowledge-injected pre-trained language model for natural language understanding and generation in e-commerce
Song Xu, Haoran Li, Peng Yuan, Yujia Wang, Youzheng Wu, Xiaodong He, Ying Liu, and Bowen Zhou. 2021 · 2021
Later among the works it cites.
QA-GNN: reasoning with language models and knowledge graphs for question answering
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Sensebert: Driving some sense into BERT
Yoav Levine, Barak Lenz, Or Dagan, Ori Ram, Dan Padnos, Or Sharir, Shai Shalev-Shwartz, Amnon Shashua, and Yoav Shoham. 2020 · 2020
Cited alongside, same era.
Retrieval-augmented generation for knowledge-intensive NLP tasks
Patrick S. H. Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, Sebastian Riedel, and Douwe Kiela. 2020 · 2020
Cited alongside, same era.
E-BERT: Efficient-yet-effective entity embeddings for BERT
Nina Poerner, Ulli Waltinger, and Hinrich Schütze. 2020 · 2020
Cited alongside, same era.
CoLAKE: Contextualized language and knowledge embedding
Tianxiang Sun, Yunfan Shao, Xipeng Qiu, Qipeng Guo, Yaru Hu, Xuanjing Huang, and Zheng Zhang. 2020 · 2020
Cited alongside, same era.
Pretrained encyclopedia: Weakly supervised knowledge-pretrained language model
Wenhan Xiong, Jingfei Du, William Yang Wang, and Veselin Stoyanov. 2020 · 2020
Cited alongside, same era.
LUKE: deep contextualized entity representations with entity-aware self-attention
Ikuya Yamada, Akari Asai, Hiroyuki Shindo, Hideaki Takeda, and Yuji Matsumoto. 2020 · 2020
Cited alongside, same era.
LIMIT-BERT : Linguistics informed multi-task BERT
Junru Zhou, Zhuosheng Zhang, Hai Zhao, and Shuailiang Zhang. 2020 · 2020
Cited alongside, same era.
Michihiro Yasunaga, Hongyu Ren, Antoine Bosselut, Percy Liang, and Jure Leskovec. 2021 · 2021
Later among the works it cites.
Bitfit: Simple parameter-efficient fine-tuning for transformer-based masked language-models
Elad Ben Zaken, Shauli Ravfogel, and Yoav Goldberg. 2021 · 2021
Later among the works it cites.
Flamingo: a visual language model for few-shot learning
Jean-Baptiste Alayrac, Jeff Donahue, Pauline Luc, Antoine Miech, Iain Barr, Yana Hasson, Karel Lenc, Arthur Mensch, Katherine Millican, Malcolm Reynolds, Roman Ring, Eliza Rutherford, Serkan Cabi, Tengda Han, Zhitao Gong, Sina Samangooei, Marianne Monteiro, Jacob L. Menick, Sebastian Borgeaud, Andy Brock, Aida Nematzadeh, Sahand Sharifzadeh, Mikolaj Binkowski, Ricardo Barreira, Oriol Vinyals, Andrew Zisserman, and Karén Simonyan. 2022 · 2022
Later among the works it cites.
Efficient hierarchical domain adaptation for pretrained language models
Alexandra Chronopoulou, Matthew E. Peters, and Jesse Dodge. 2022 · 2022
Later among the works it cites.
Delta tuning: A comprehensive study of parameter efficient methods for pre-trained language models
Ning Ding, Yujia Qin, Guang Yang, Fuchao Wei, Zonghan Yang, Yusheng Su, Shengding Hu, Yulin Chen, Chi-Min Chan, Weize Chen, Jing Yi, Weilin Zhao, Xiaozhi Wang, Zhiyuan Liu, Hai-Tao Zheng, Jianfei Chen, Yang Liu, Jie Tang, Juanzi Li, and Maosong Sun. 2022 · 2022
Later among the works it cites.
KALA: knowledge-augmented language model adaptation
Minki Kang, Jinheon Baek, and Sung Ju Hwang. 2022 · 2022
Later among the works it cites.
A simple but effective pluggable entity lookup table for pre-trained language models
Deming Ye, Yankai Lin, Peng Li, Maosong Sun, and Zhiyuan Liu. 2022 · 2022
Later among the works it cites.
A survey of knowledge-intensive nlp with pre-trained language models
Da Yin, Li Dong, Hao Cheng, Xiaodong Liu, Kai-Wei Chang, Furu Wei, and Jianfeng Gao. 2022 · 2022
Later among the works it cites.
Can lms learn new entities from descriptions? challenges in propagating injected knowledge
Yasumasa Onoe, Michael J. Q. Zhang, Shankar Padmanabhan, Greg Durrett, and Eunsol Choi. 2023 · 2023
Closest in time.
Plug-and-play document modules for pre-trained models
Chaojun Xiao, Zhengyan Zhang, Xu Han, Chi-Min Chan, Yankai Lin, Zhiyuan Liu, Xiangyang Li, Zhonghua Li, Zhao Cao, and Maosong Sun. 2023 · 2023
Closest in time.
Small models are valuable plug-ins for large language models
Canwen Xu, Yichong Xu, Shuohang Wang, Yang Liu, Chenguang Zhu, and Julian McAuley. 2023 · 2023
Closest in time.
Augmentation-adapted retriever improves generalization of language models as a zero-shot plug-in
Zichun Yu, Chenyan Xiong, Shi Yu, and Zhiyuan Liu. 2023 · 2023
Closest in time.