Fetching the paper…
Reading the bibliography…
Pre-trained Language Models (PLMs) which are trained on large text corpus via self-supervised learning method, have yielded promising performance on various tasks in Natural Language Processing (NLP).
W. Yu, C. Zhu, L. Qin, Z. Zhang, T. Zhao, and M. Jiang, “Diversifying content generation for commonsense reasoning with mixture of knowledge graph experts,” in ACL , 2022, pp. 1896–1906
1906
Earlier work this paper cites.
A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. Kaiser, and I. Polosukhin, “Attention is all you need,” in NIPS , 2017, pp. 5998–6008
2017
Earlier work this paper cites.
E. Grave, A. Joulin, and N. Usunier, “Improving neural language models with a continuous cache,” in ICLR , 2017
2017
Earlier work this paper cites.
Q. Wang, Z. Mao, B. Wang, and L. Guo, “Knowledge graph embedding: A survey of approaches and applications,” IEEE Transactions on Knowledge and Data Engineering , vol. 29, no. 12, pp. 2724–2743, 2017
2017
Earlier work this paper cites.
A. Radford, K. Narasimhan, T. Salimans, I. Sutskever et al. , “Improving language understanding by generative pre-training,” 2018
2018
Earlier work this paper cites.
M. E. Peters, M. Neumann, M. Iyyer, M. Gardner, C. Clark, K. Lee, and L. Zettlemoyer, “Deep contextualized word representations,” in NAACL-HLT , 2018, pp. 2227–2237
2018
Earlier work this paper cites.
J. Howard and S. Ruder, “Universal language model fine-tuning for text classification,” in ACL , 2018, pp. 328–339
2018
Earlier work this paper cites.
S. Rajbhandari, J. Rasley, O. Ruwase, and Y. He, “Zero: memory optimizations toward training trillion parameter models,” in SC20: International Conference for High Performance Computing, Networking, Storage and Analysis . IEEE, 2020, pp. 1–16
2018
Earlier work this paper cites.
J. Devlin, M. Chang, K. Lee, and K. Toutanova, “BERT: pre-training of deep bidirectional transformers for language understanding,” in NAACL-HLT , 2019, pp. 4171–4186
2019
Earlier work this paper cites.
N. F. Liu, M. Gardner, Y. Belinkov, M. E. Peters, and N. A. Smith, “Linguistic knowledge and transferability of contextual representations,” in NAACL-HLT , 2019, pp. 1073–1094
2019
Earlier work this paper cites.
Y. Belinkov and J. R. Glass, “Analysis methods in neural language processing: A survey,” in NAACL-HLT , J. Burstein, C. Doran, and T. Solorio, Eds., 2019, pp. 3348–3354
2019
Earlier work this paper cites.
F. Petroni, T. Rocktäschel, S. Riedel, P. S. H. Lewis, A. Bakhtin, Y. Wu, and A. H. Miller, “Language models as knowledge bases?” in EMNLP-IJCNLP , 2019, pp. 2463–2473
2019
Earlier work this paper cites.
L. Dong, N. Yang, W. Wang, F. Wei, X. Liu, Y. Wang, J. Gao, M. Zhou, and H. Hon, “Unified language model pre-training for natural language understanding and generation,” in NeurIPS , 2019, pp. 13 042–13 054
2019
Earlier work this paper cites.
I. Beltagy, K. Lo, and A. Cohan, “Scibert: A pretrained language model for scientific text,” in EMNLP-IJCNLP , 2019, pp. 3613–3618
2019
Earlier work this paper cites.
2019
Earlier work this paper cites.
Z. Zhang, X. Han, Z. Liu, X. Jiang, M. Sun, and Q. Liu, “ERNIE: enhanced language representation with informative entities,” in ACL , 2019, pp. 1441–1451
2019
Earlier work this paper cites.
M. E. Peters, M. Neumann, R. L. L. IV, R. Schwartz, V. Joshi, S. Singh, and N. A. Smith, “Knowledge enhanced contextual word representations,” in EMNLP-IJCNLP , 2019, pp. 43–54
2019
Earlier work this paper cites.
2019
Earlier work this paper cites.
E. Dinan, S. Roller, K. Shuster, A. Fan, M. Auli, and J. Weston, “Wizard of wikipedia: Knowledge-powered conversational agents,” in ICLR , 2019
2019
Earlier work this paper cites.
2019
Earlier work this paper cites.
C. Raffel, N. Shazeer, A. Roberts, K. Lee, S. Narang, M. Matena, Y. Zhou, W. Li, and P. J. Liu, “Exploring the limits of transfer learning with a unified text-to-text transformer,” Journal of Machine Learning Research , vol. 21, pp. 140:1–140:67, 2020
2020
Earlier work this paper cites.
I. Vulic, E. M. Ponti, R. Litschko, G. Glavas, and A. Korhonen, “Probing pretrained language models for lexical semantics,” in EMNLP , 2020, pp. 7222–7240
2020
Earlier work this paper cites.
A. Talmor, Y. Elazar, Y. Goldberg, and J. Berant, “olmpics - on what language model pre-training captures,” Transactions of the Association for Computational Linguistics , vol. 8, pp. 743–758, 2020
2020
Earlier work this paper cites.
M. Lewis, Y. Liu, N. Goyal, M. Ghazvininejad, A. Mohamed, O. Levy, V. Stoyanov, and L. Zettlemoyer, “BART: denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension,” in ACL , 2020, pp. 7871–7880
2020
Earlier work this paper cites.
T. Shin, Y. Razeghi, R. L. L. IV, E. Wallace, and S. Singh, “Autoprompt: Eliciting knowledge from language models with automatically generated prompts,” in EMNLP , 2020, pp. 4222–4235
2020
Earlier work this paper cites.
A. Lauscher, I. Vulic, E. M. Ponti, A. Korhonen, and G. Glavas, “Specializing unsupervised pretraining models for word-level semantic similarity,” in COLING , 2020, pp. 1371–1383
2020
Earlier work this paper cites.
Y. Levine, B. Lenz, O. Dagan, O. Ram, D. Padnos, O. Sharir, S. Shalev-Shwartz, A. Shashua, and Y. Shoham, “Sensebert: Driving some sense into BERT,” in ACL , 2020, pp. 4656–4667
2020
Earlier work this paper cites.
H. Tian, C. Gao, X. Xiao, H. Liu, B. He, H. Wu, H. Wang, and F. Wu, “SKEP: sentiment knowledge enhanced pre-training for sentiment analysis,” in ACL , 2020, pp. 4067–4076
2020
Earlier work this paper cites.
J. Zhou, Z. Zhang, H. Zhao, and S. Zhang, “LIMIT-BERT : Linguistics informed multi-task BERT,” in EMNLP , 2020, pp. 4450–4461
2020
Earlier work this paper cites.
U. Khandelwal, O. Levy, D. Jurafsky, L. Zettlemoyer, and M. Lewis, “Generalization through memorization: Nearest neighbor language models,” in ICLR , 2020
2020
Earlier work this paper cites.
K. Guu, K. Lee, Z. Tung, P. Pasupat, and M. Chang, “Retrieval augmented language model pre-training,” in ICML , 2020, pp. 3929–3938
2020
Earlier work this paper cites.
S. Murty, P. W. Koh, and P. Liang, “Expbert: Representation engineering with natural language explanations,” in ACL , 2020, pp. 2106–2113
2020
Earlier work this paper cites.
J. Lee, W. Yoon, S. Kim, D. Kim, S. Kim, C. H. So, and J. Kang, “Biobert: a pre-trained biomedical language representation model for biomedical text mining,” Bioinformatics , vol. 36, no. 4, pp. 1234–1240, 2020
2020
Earlier work this paper cites.
K. Lo, L. L. Wang, M. Neumann, R. Kinney, and D. S. Weld, “S2ORC: the semantic scholar open research corpus,” in ACL , 2020, pp. 4969–4983
2020
Earlier work this paper cites.
W. Xiong, J. Du, W. Y. Wang, and V. Stoyanov, “Pretrained encyclopedia: Weakly supervised knowledge-pretrained language model,” in ICLR , 2020
2020
Earlier work this paper cites.
I. Yamada, A. Asai, H. Shindo, H. Takeda, and Y. Matsumoto, “LUKE: deep contextualized entity representations with entity-aware self-attention,” in EMNLP , 2020, pp. 6442–6454
2020
Earlier work this paper cites.
T. Févry, L. B. Soares, N. FitzGerald, E. Choi, and T. Kwiatkowski, “Entities as experts: Sparse memory access with entity supervision,” in EMNLP , 2020, pp. 4937–4951
2020
Earlier work this paper cites.
W. Liu, P. Zhou, Z. Zhao, Z. Wang, Q. Ju, H. Deng, and P. Wang, “K-bert: Enabling language representation with knowledge graph,” in AAAI , vol. 34, no. 03, 2020, pp. 2901–2908
2020
Cited alongside, same era.
T. Sun, Y. Shao, X. Qiu, Q. Guo, Y. Hu, X. Huang, and Z. Zhang, “Colake: Contextualized language and knowledge embedding,” in COLING , 2020, pp. 3660–3670
2020
Cited alongside, same era.
2020
Cited alongside, same era.
P. Lewis, E. Perez, A. Piktus, F. Petroni, V. Karpukhin, N. Goyal, H. Küttler, M. Lewis, W.-t. Yih, T. Rocktäschel et al. , “Retrieval-augmented generation for knowledge-intensive nlp tasks,” Advances in Neural Information Processing Systems , vol. 33, pp. 9459–9474, 2020
2020
Cited alongside, same era.
2022
Closest in time.
Y. Gu, X. Han, Z. Liu, and M. Huang, “PPT: pre-trained prompt tuning for few-shot learning,” in ACL , 2022, pp. 8410–8423
2022
Closest in time.
N. Zhang, L. Li, X. Chen, S. Deng, Z. Bi, C. Tan, F. Huang, and H. Chen, “Differentiable prompt makes pre-trained language models better few-shot learners,” in ICLR , 2022
2022
Closest in time.
N. Ding, S. Hu, W. Zhao, Y. Chen, Z. Liu, H. Zheng, and M. Sun, “Openprompt: An open-source framework for prompt-learning,” in ACL , 2022, pp. 105–113
2022
Closest in time.
S. Hu, N. Ding, H. Wang, Z. Liu, J. Wang, J. Li, W. Wu, and M. Sun, “Knowledgeable prompt-tuning: Incorporating knowledge into prompt verbalizer for text classification,” in ACL , 2022, pp. 2225–2240
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
B. Kim, J. Ahn, and G. Kim, “Sequential latent knowledge selection for knowledge-grounded dialogue,” in ICLR , 2020
2020
Cited alongside, same era.
H. Ji, P. Ke, S. Huang, F. Wei, and M. Huang, “Generating commonsense explanation by extracting bridge concepts from reasoning paths,” in AACL-IJCNLP , 2020, pp. 248–257
2020
Cited alongside, same era.
2020
Cited alongside, same era.
J. Guan, F. Huang, M. Huang, Z. Zhao, and X. Zhu, “A knowledge-enhanced pretraining model for commonsense story generation,” Transactions of the Association for Computational Linguistics , vol. 8, pp. 93–108, 2020
2020
Cited alongside, same era.
H. Ji, P. Ke, S. Huang, F. Wei, X. Zhu, and M. Huang, “Language generation with multi-hop reasoning on commonsense knowledge graph,” in EMNLP , 2020, pp. 725–736
2020
Cited alongside, same era.
2021
Cited alongside, same era.
Y. Liu, Y. Wan, L. He, H. Peng, and S. Y. Philip, “Kg-bart: Knowledge graph-augmented bart for generative commonsense reasoning,” in AAAI , vol. 35, no. 7, 2021, pp. 6418–6425
2021
Cited alongside, same era.
2021
Cited alongside, same era.
2022
Closest in time.
X. Chen, N. Zhang, X. Xie, S. Deng, Y. Yao, C. Tan, F. Huang, L. Si, and H. Chen, “Knowprompt: Knowledge-aware prompt-tuning with synergistic optimization for relation extraction,” in ACM , 2022, pp. 2778–2788
2022
Closest in time.
H. Ye, N. Zhang, S. Deng, X. Chen, H. Chen, F. Xiong, X. Chen, and H. Chen, “Ontology-enhanced prompt-tuning for few-shot learning,” in ACM , 2022, pp. 778–787
2022
Closest in time.
W. Yu, C. Zhu, Z. Li, Z. Hu, Q. Wang, H. Ji, and M. Jiang, “A survey of knowledge-enhanced text generation,” ACM Computing Surveys (CSUR) , 2022
2022
Closest in time.
Y. Xu, C. Zhu, S. Wang, S. Sun, H. Cheng, X. Liu, J. Gao, P. He, M. Zeng, and X. Huang, “Human parity on commonsenseqa: Augmenting self-attention with external attention,” in IJCAI , 2022, pp. 2762–2768
2022
Closest in time.
Q. Chen, F. Li, G. Xu, M. Yan, J. Zhang, and Y. Zhang, “Dictbert: Dictionary description knowledge enhanced language model pre-training via contrastive learning,” in IJCAI , 2022, pp. 4086–4092
2022
Closest in time.
W. Cui and X. Chen, “Enhancing natural language representation with large-scale out-of-domain commonsense,” in ACL , 2022, pp. 1746–1756
2022
Closest in time.
Y. Yao, S. Huang, L. Dong, F. Wei, H. Chen, and N. Zhang, “Kformer: Knowledge injection in transformer feed-forward layers,” in NLPCC , vol. 13551, 2022, pp. 131–143
2022
Closest in time.
S. Wang, Y. Xu, Y. Fang, Y. Liu, S. Sun, R. Xu, C. Zhu, and M. Zeng, “Training data is more valuable than you think: A simple and effective method by retrieving from training data,” in ACL , 2022, pp. 3170–3179
2022
Closest in time.
B. Oguz, X. Chen, V. Karpukhin, S. Peshterliev, D. Okhonko, M. S. Schlichtkrull, S. Gupta, Y. Mehdad, and S. Yih, “Unik-qa: Unified representations of structured and unstructured knowledge for open-domain question answering,” in NAACL , 2022, pp. 1535–1546
2022
Closest in time.
K. Ma, H. Cheng, X. Liu, E. Nyberg, and J. Gao, “Open domain question answering with a unified knowledge interface,” in ACL , 2022, pp. 1605–1620
2022
Closest in time.
B. Xue and L. Zou, “Knowledge graph quality management: a comprehensive survey,” IEEE Transactions on Knowledge and Data Engineering , 2022
2022
Closest in time.
L. Fang, Y. Luo, K. Feng, K. Zhao, and A. Hu, “A knowledge-enriched ensemble method for word embedding and multi-sense embedding,” IEEE Transactions on Knowledge and Data Engineering , 2022
2022
Closest in time.
Q. Li, D. Wang, S. F. K. Song, Y. Zhang, and G. Yu, “Oerl: Enhanced representation learning via open knowledge graphs,” IEEE Transactions on Knowledge and Data Engineering , 2022
2022
Closest in time.
Y. Zhao, H. Zhou, A. Zhang, R. Xie, Q. Li, and F. Zhuang, “Connecting embeddings based on multiplex relational graph attention networks for knowledge graph entity typing,” IEEE Transactions on Knowledge and Data Engineering , 2022
2022
Closest in time.
2022
Closest in time.
T. Zhang, C. Wang, N. Hu, M. Qiu, C. Tang, X. He, and J. Huang, “Dkplm: Decomposable knowledge-enhanced pre-trained language model for natural language understanding,” in AAAI , vol. 36, no. 10, 2022, pp. 11 703–11 711
2022
Closest in time.
2022
Closest in time.
D. Emelin, D. Bonadiman, S. Alqahtani, Y. Zhang, and S. Mansour, “Injecting domain knowledge in language models for task-oriented dialogue systems,” in EMNLP , 2022
2022
Closest in time.
Q. Dong, Y. Liu, S. Cheng, S. Wang, Z. Cheng, S. Niu, and D. Yin, “Incorporating explicit knowledge in pre-trained language models for passage re-ranking,” in SIGIR , 2022, pp. 1490–1501
2022
Closest in time.
Y. Sun, Q. Shi, L. Qi, and Y. Zhang, “Jointlk: Joint reasoning with language models and knowledge graphs for commonsense question answering,” in NAACL , 2022, pp. 5049–5060
2022
Closest in time.
X. Zhang, A. Bosselut, M. Yasunaga, H. Ren, P. Liang, C. D. Manning, and J. Leskovec, “Greaselm: Graph reasoning enhanced language models,” in ICLR , 2022
2022
Closest in time.
D. Yu, C. Zhu, Y. Yang, and M. Zeng, “JAKET: joint pre-training of knowledge graph and language understanding,” in AAAI , 2022, pp. 11 630–11 638
2022
Closest in time.
X. Liu, D. Liu, B. Yang, H. Zhang, J. Ding, W. Yao, W. Luo, H. Zhang, and J. Su, “Kgr4: Retrieval, retrospect, refine and rethink for commonsense generation,” in AAAI , vol. 36, no. 10, 2022, pp. 11 029–11 037
2022
Closest in time.
D. Yu, C. Zhu, Y. Fang, W. Yu, S. Wang, Y. Xu, X. Ren, Y. Yang, and M. Zeng, “Kg-fid: Infusing knowledge graph in fusion-in-decoder for open-domain question answering,” in ACL , 2022, pp. 4961–4974
2022
Closest in time.
2022
Closest in time.
S. Borgeaud, A. Mensch, J. Hoffmann, T. Cai, E. Rutherford, K. Millican, G. B. Van Den Driessche, J.-B. Lespiau, B. Damoc, A. Clark et al. , “Improving language models by retrieving from trillions of tokens,” in ICML , 2022, pp. 2206–2240
2022
Closest in time.
2022
Closest in time.
Y. Li, B. Peng, Y. Shen, Y. Mao, L. Liden, Z. Yu, and J. Gao, “Knowledge-grounded dialogue generation with a unified knowledge representation,” in NAACL , 2022, pp. 206–218
2022
Closest in time.
R. Sridhar and D. Yang, “Explaining toxic text via knowledge enhanced text generation,” in NAACL , 2022, pp. 811–826
2022
Closest in time.
D. Varshney, A. Prabhakar, and A. Ekbal, “Commonsense and named entity aware knowledge grounded dialogue generation,” in NAACL , 2022, pp. 1322–1335
2022
Closest in time.
Y. Qin, J. Zhang, Y. Lin, Z. Liu, P. Li, M. Sun, and J. Zhou, “ELLE: efficient lifelong pre-training for emerging data,” in ACL , 2022, pp. 2789–2810
2022
Closest in time.