2021

Triple Classification for Scholarly Knowledge Graph Completion

Jaradeh, Mohamad Yaser, Singh, Kuldeep, Stocker, Markus et al.

Understand

Scholarly Knowledge Graphs (KGs) provide a rich source of structured information representing knowledge encoded in scientific publications.

With the sheer volume of published scientific literature comprising a plethora of inhomogeneous entities and relations to describe scientific concepts, these KGs are inherently incomplete.
We present exBERT, a method for leveraging pre-trained transformer language models to perform scholarly knowledge graph completion.
We model triples of a knowledge graph as text and perform triple classification (i.e., belongs to KG or not).

Built on

KG-BERT: BERT for Knowledge Graph Completion
Original
Liang Yao, Chengsheng Mao, and Yuan Luo. 2019 · 1909
Earlier work this paper cites.
The Unified Medical Language System (UMLS): integrating biomedical terminology
Olivier Bodenreider. 2004 · 2004
Earlier work this paper cites.
RDF primer
Frank Manola, Eric Miller, Brian McBride, et al · 2004
Earlier work this paper cites.
Dbpedia: A nucleus for a web of open data
Sören Auer, Christian Bizer, Georgi Kobilarov, Jens Lehmann, Richard Cyganiak, and Zachary Ives. 2007 · 2007
Earlier work this paper cites.
The fourth paradigm: data-intensive scientific discovery
Anthony JG Hey, Stewart Tansley, Kristin Michele Tolle, et al · 2009
Earlier work this paper cites.
PairRE: Knowledge Graph Embeddings via Paired Relation Vectors
Original
Linlin Chao, Jianshan He, Taifeng Wang, and Wei Chu. 2021 · 2011
Earlier work this paper cites.
A Three-Way Model for Collective Learning on Multi-Relational Data (ICML’11) . Omnipress, 809–816
Maximilian Nickel, Volker Tresp, and Hans-Peter Kriegel. 2011 · 2011
Earlier work this paper cites.
Translating Embeddings for Modeling Multi-relational Data. In Advances in Neural Information Processing Systems , Vol. 26
Antoine Bordes, Nicolas Usunier, Alberto Garcia-Duran, Jason Weston, and Oksana Yakhnenko. 2013 · 2013
Earlier work this paper cites.
Distributed Representations of Words and Phrases and Their Compositionality. In Proceedings of the 26th International Conference on Neural Information Processing Systems - Volume 2 (NIPS’13) . 3111–3119
Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg Corrado, and Jeffrey Dean. 2013 · 2013
Earlier work this paper cites.
Reasoning with Neural Tensor Networks for Knowledge Base Completion (NIPS’13) . 926–934
Richard Socher, Danqi Chen, Christopher D. Manning, and Andrew Y. Ng. 2013 · 2013
Earlier work this paper cites.
GloVe: Global Vectors for Word Representation. In EMNLP . ACL, 1532–1543
Jeffrey Pennington, Richard Socher, and Christopher Manning. 2014 · 2014
Earlier work this paper cites.
Wikidata: A Free Collaborative Knowledgebase
Denny Vrandečić and Markus Krötzsch. 2014 · 2014
Earlier work this paper cites.
Knowledge Graph Embedding by Translating on Hyperplanes (AAAI’14) . 1112–1119
Zhen Wang, Jianwen Zhang, Jianlin Feng, and Zheng Chen. 2014 · 2014
Earlier work this paper cites.
Embedding entities and relations for learning and inference in knowledge bases
Original
Bishan Yang, Wen-tau Yih, Xiaodong He, Jianfeng Gao, and Li Deng. 2014 · 2014
Earlier work this paper cites.
Learning to Represent Knowledge Graphs with Gaussian Embedding. In Proceedings of the 24th ACM International on Conference on Information and Knowledge Management . Association for Computing Machinery, 623–632
Shizhu He, Kang Liu, Guoliang Ji, and Jun Zhao. 2015 · 2015
Earlier work this paper cites.
Knowledge Graph Embedding via Dynamic Mapping Matrix. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th IJCNLP (Volume 1: Long Papers) . ACL, 687–696
Guoliang Ji, Shizhu He, Liheng Xu, Kang Liu, and Jun Zhao. 2015 · 2015
Earlier work this paper cites.

Similar

Learning Entity and Relation Embeddings for Knowledge Graph Completion (AAAI’15)
Yankai Lin, Zhiyuan Liu, Maosong Sun, Yang Liu, and Xuan Zhu. 2015 · 2015
Cited alongside, same era.
Enriching Word Vectors with Subword Information
Original
Piotr Bojanowski, Edouard Grave, Armand Joulin, and Tomas Mikolov. 2016 · 2016
Cited alongside, same era.
Knowledge Graph Completion with Adaptive Sparse Transfer Matrix. In Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence (AAAI’16) . 985–991
Guoliang Ji, Kang Liu, Shizhu He, and Jun Zhao. 2016 · 2016
Cited alongside, same era.
Holographic embeddings of knowledge graphs. In AAAI , Vol. 30
Maximilian Nickel, Lorenzo Rosasco, and Tomaso Poggio. 2016 · 2016
Cited alongside, same era.
Text-enhanced representation learning for knowledge graph. In IJCAI . 4–17
Zhigang Wang, Juanzi Li, Zhiyuan Liu, and Jie Tang. 2016 · 2016
Cited alongside, same era.
The FAIR Guiding Principles for scientific data management and stewardship
Mark D Wilkinson, Michel Dumontier, IJsbrand Jan Aalbersberg, Gabrielle Appleton, Myles Axton, Arie Baak, Niklas Blomberg, Jan-Willem Boiten, Luiz Bonino da Silva Santos, Philip E Bourne, et al · 2016
Cited alongside, same era.
TransG : A Generative Model for Knowledge Graph Embedding. In ACL (Volume 1: Long Papers) . ACL, 2316–2325
Han Xiao, Minlie Huang, and Xiaoyan Zhu. 2016 · 2016
Cited alongside, same era.
KBQA: Learning Question Answering over QA Corpora and Knowledge Bases
Wanyun Cui, Yanghua Xiao, Haixun Wang, Yangqiu Song, Seung-won Hwang, and Wei Wang. 2017 · 2017
Cited alongside, same era.
ProjE: Embedding Projection for Knowledge Graph Completion (AAAI’17) . 1236–1242
Baoxu Shi and Tim Weninger. 2017 · 2017
Cited alongside, same era.
Knowledge Graph Embedding: A Survey of Approaches and Applications
Quan Wang, Zhendong Mao, Bin Wang, and Li Guo. 2017 · 2017
Cited alongside, same era.
Science and engineering publication output trends: 2014 shows rise of developing country output while developed countries dominate highly cited publications
Karen E White, Carol Robbins, Beethika Khan, and Christina Freyman. 2017 · 2017
Cited alongside, same era.
Contextual String Embeddings for Sequence Labeling. In COLING 2018, 27th International Conference on Computational Linguistics . ACL, 1638–1649
Alan Akbik, Duncan Blythe, and Roland Vollgraf. 2018 · 2018
Cited alongside, same era.

Then

SimplE Embedding for Link Prediction in Knowledge Graphs. In Advances in Neural Information Processing Systems , Vol. 31
Seyed Mehran Kazemi and David Poole. 2018 · 2018
Later among the works it cites.
A Novel Embedding Model for Knowledge Base Completion Based on Convolutional Neural Network. In NAACL-HLT, Volume 2 (Short Papers) . ACL, 327–333
Dai Quoc Nguyen, Tu Dinh Nguyen, Dat Quoc Nguyen, and Dinh Phung. 2018 · 2018
Later among the works it cites.
RotatE: Knowledge Graph Embedding by Relational Rotation in Complex Space. In International Conference on Learning Representations
Zhiqing Sun, Zhi-Hong Deng, Jian-Yun Nie, and Jian Tang. 2018 · 2018
Later among the works it cites.
SciBERT: A Pretrained Language Model for Scientific Text. In EMNLP-IJCNLP . ACL, 3615–3620
Iz Beltagy, Kyle Lo, and Arman Cohan. 2019 · 2019
Later among the works it cites.
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In NAACL-HLT . ACL, 4171–4186
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019 · 2019
Later among the works it cites.
The microsoft academic knowledge graph: a linked data source with 8 billion triples of scholarly data. In International Semantic Web Conference . Springer, 113–129
Michael Färber. 2019 · 2019
Later among the works it cites.
Open Research Knowledge Graph. In Proceedings of the 10th International Conference on Knowledge Capture . ACM
Mohamad Yaser Jaradeh, Allard Oelen, Kheir Eddine Farfar, Manuel Prinz, Jennifer D'Souza, Gábor Kismihók, Markus Stocker, and Sören Auer. 2019 · 2019
Later among the works it cites.
Soft marginal transe for scholarly knowledge graph completion
Original
Mojtaba Nayyeri, Sahar Vahdati, Jens Lehmann, and Hamed Shariat Yazdi. 2019 · 2019
Later among the works it cites.
ERNIE: Enhanced Language Representation with Informative Entities. ACL, 1441–1451
Original
Zhengyan Zhang, Xu Han, Zhiyuan Liu, Xin Jiang, Maosong Sun, and Qun Liu. 2019 · 2019
Later among the works it cites.
AI-KG: An Automatically Generated Knowledge Graph of Artificial Intelligence. In The Semantic Web – ISWC 2020 . Springer International Publishing, 127–143
Danilo Dessì, Francesco Osborne, Diego Reforgiato Recupero, Davide Buscaldi, Enrico Motta, and Harald Sack. 2020 · 2020
Later among the works it cites.
Question Answering on Scholarly Knowledge Graphs. In International Conference on Theory and Practice of Digital Libraries . Springer, 19–32
Mohamad Yaser Jaradeh, Markus Stocker, and Sören Auer. 2020 · 2020
Later among the works it cites.
Evaluating the impact of knowledge graph context on entity disambiguation models. In CIKM . 2157–2160
Isaiah Onando Mulang’, Kuldeep Singh, Chaitali Prabhu, Abhishek Nadgeri, Johannes Hoffart, and Jens Lehmann. 2020 · 2020
Later among the works it cites.
Composition-based Multi-Relational Graph Convolutional Networks. In International Conference on Learning Representations
Shikhar Vashishth, Soumya Sanyal, Vikram Nitin, and Partha Talukdar. 2020 · 2020
Later among the works it cites.
PyKEEN 1.0: A Python Library for Training and Evaluating Knowledge Graph Embeddings
Mehdi Ali, Max Berrendorf, Charles Tapley Hoyt, Laurent Vermue, Sahand Sharifzadeh, Volker Tresp, and Jens Lehmann. 2021 · 2021
Closest in time.
Complex embeddings for simple link prediction. In International Conference on Machine Learning . PMLR, 2071–2080
Théo Trouillon, Johannes Welbl, Sebastian Riedel, Éric Gaussier, and Guillaume Bouchard. 2016 · 2080
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…