Fetching the paper…
Reading the bibliography…
Graph pre-training has been concentrated on graph-level tasks involving small graphs (e.g., molecular graphs) or learning node representations on a fixed graph.
Language models are few-shot learners
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al · 1901
Earlier work this paper cites.
Automating the construction of internet portals with machine learning
Andrew Kachites McCallum, Kamal Nigam, Jason Rennie, and Kristie Seymore. 2000 · 2000
Earlier work this paper cites.
Collective classification in network data
Prithviraj Sen, Galileo Namata, Mustafa Bilgic, Lise Getoor, Brian Galligher, and Tina Eliassi-Rad. 2008 · 2008
Earlier work this paper cites.
Extracting and composing robust features with denoising autoencoders. In Proceedings of the 25th international conference on Machine learning . 1096–1103
Pascal Vincent, Hugo Larochelle, Yoshua Bengio, and Pierre-Antoine Manzagol. 2008 · 2008
Earlier work this paper cites.
Stacked denoising autoencoders: Learning useful representations in a deep network with a local denoising criterion
Pascal Vincent, Hugo Larochelle, Isabelle Lajoie, Yoshua Bengio, Pierre-Antoine Manzagol, and Léon Bottou. 2010 · 2010
Earlier work this paper cites.
Line: Large-scale information network embedding. In Proceedings of the 24th international conference on world wide web . 1067–1077
Jian Tang, Meng Qu, Mingzhe Wang, Ming Zhang, Jun Yan, and Qiaozhu Mei. 2015 · 2015
Earlier work this paper cites.
Xgboost: A scalable tree boosting system. In Proceedings of the 22nd acm sigkdd international conference on knowledge discovery and data mining . 785–794
Tianqi Chen and Carlos Guestrin. 2016 · 2016
Earlier work this paper cites.
Semi-supervised classification with graph convolutional networks
Thomas N Kipf and Max Welling. 2016a · 2016
Earlier work this paper cites.
Variational graph auto-encoders
Thomas N Kipf and Max Welling. 2016b · 2016
Earlier work this paper cites.
Inductive representation learning on large graphs
Will Hamilton, Zhitao Ying, and Jure Leskovec. 2017 · 2017
Earlier work this paper cites.
Attention is all you need
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017 · 2017
Earlier work this paper cites.
Graph attention networks
Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio, and Yoshua Bengio. 2017 · 2017
Earlier work this paper cites.
Mgae: Marginalized graph autoencoder for graph clustering. In Proceedings of the 2017 ACM on Conference on Information and Knowledge Management . 889–898
Chun Wang, Shirui Pan, Guodong Long, Xingquan Zhu, and Jing Jiang. 2017 · 2017
Earlier work this paper cites.
Fastgcn: fast learning with graph convolutional networks via importance sampling
Jie Chen, Tengfei Ma, and Cao Xiao. 2018 · 2018
Earlier work this paper cites.
Bert: Pre-training of deep bidirectional transformers for language understanding
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018 · 2018
Earlier work this paper cites.
Deeper insights into graph convolutional networks for semi-supervised learning. In Proceedings of the AAAI conference on artificial intelligence , Vol. 32
Qimai Li, Zhichao Han, and Xiao-Ming Wu. 2018 · 2018
Earlier work this paper cites.
Deep graph infomax
Petar Veličković, William Fedus, William L Hamilton, Pietro Liò, Yoshua Bengio, and R Devon Hjelm. 2018 · 2018
Earlier work this paper cites.
How powerful are graph neural networks?
Keyulu Xu, Weihua Hu, Jure Leskovec, and Stefanie Jegelka. 2018 · 2018
Earlier work this paper cites.
Cluster-gcn: An efficient algorithm for training deep and large graph convolutional networks. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining . 257–266
Wei-Lin Chiang, Xuanqing Liu, Si Si, Yang Li, Samy Bengio, and Cho-Jui Hsieh. 2019 · 2019
Earlier work this paper cites.
How robust are graph neural networks to structural noise?
James Fox and Sivasankaran Rajamanickam. 2019 · 2019
Earlier work this paper cites.
Strategies for pre-training graph neural networks
Weihua Hu, Bowen Liu, Joseph Gomes, Marinka Zitnik, Percy Liang, Vijay Pande, and Jure Leskovec. 2019 · 2019
Earlier work this paper cites.
Deep graph library: A graph-centric, highly-performant package for graph neural networks
Minjie Wang, Da Zheng, Zihao Ye, Quan Gan, Mufei Li, Xiang Song, Jinjing Zhou, Chao Ma, Lingfan Yu, Yu Gai, et al · 2019
Earlier work this paper cites.
Graphsaint: Graph sampling based inductive learning method
Hanqing Zeng, Hongkuan Zhou, Ajitesh Srivastava, Rajgopal Kannan, and Viktor Prasanna. 2019 · 2019
Cited alongside, same era.
On the bottleneck of graph neural networks and its practical implications
Uri Alon and Eran Yahav. 2020 · 2020
Cited alongside, same era.
A generalization of transformer networks to graphs
Vijay Prakash Dwivedi and Xavier Bresson. 2020 · 2020
Cited alongside, same era.
Bootstrap your own latent-a new approach to self-supervised learning
Jean-Bastien Grill, Florian Strub, Florent Altché, Corentin Tallec, Pierre Richemond, Elena Buchatskaya, Carl Doersch, Bernardo Avila Pires, Zhaohan Guo, Mohammad Gheshlaghi Azar, et al · 2020
Cited alongside, same era.
Contrastive multi-view representation learning on graphs. In International conference on machine learning . PMLR, 4116–4126
Kaveh Hassani and Amir Hosein Khasahmadi. 2020 · 2020
Cited alongside, same era.
Improving the training of graph neural networks with consistency regularization
Chenhui Zhang, Yufei He, Yukuo Cen, Zhenyu Hou, and Jie Tang. 2021a · 2021
Later among the works it cites.
Graph-less neural networks: Teaching old mlps new tricks via distillation
Shichang Zhang, Yozen Liu, Yizhou Sun, and Neil Shah. 2021c · 2021
Later among the works it cites.
Gophormer: Ego-graph transformer for node classification
Jianan Zhao, Chaozhuo Li, Qianlong Wen, Yiqi Wang, Yuming Liu, Hao Sun, Xing Xie, and Yanfang Ye. 2021 · 2021
Later among the works it cites.
Structure-aware transformer for graph representation learning. In International Conference on Machine Learning . PMLR, 3469–3489
Dexiong Chen, Leslie O’Bray, and Karsten Borgwardt. 2022 · 2022
Later among the works it cites.
Long range graph benchmark
Vijay Prakash Dwivedi, Ladislav Rampášek, Michael Galkin, Ali Parviz, Guy Wolf, Anh Tuan Luu, and Dominique Beaini. 2022 · 2022
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Deberta: Decoding-enhanced bert with disentangled attention
Pengcheng He, Xiaodong Liu, Jianfeng Gao, and Weizhu Chen. 2020 · 2020
Cited alongside, same era.
PSGraph: How Tencent trains extremely large-scale graphs with Spark?. In 2020 IEEE 36th International Conference on Data Engineering (ICDE) . IEEE, 1549–1557
Jiawei Jiang, Pin Xiao, Lele Yu, Xiaosen Li, Jiefeng Cheng, Xupeng Miao, Zhipeng Zhang, and Bin Cui. 2020 · 2020
Cited alongside, same era.
Gcc: Graph contrastive coding for graph neural network pre-training. In Proceedings of the 26th ACM SIGKDD international conference on knowledge discovery & data mining . 1150–1160
Jiezhong Qiu, Qibin Chen, Yuxiao Dong, Jing Zhang, Hongxia Yang, Ming Ding, Kuansan Wang, and Jie Tang. 2020 · 2020
Cited alongside, same era.
Self-supervised graph transformer on large-scale molecular data
Yu Rong, Yatao Bian, Tingyang Xu, Weiyang Xie, Ying Wei, Wenbing Huang, and Junzhou Huang. 2020 · 2020
Cited alongside, same era.
Approximating spectral clustering via sampling: a review
Nicolas Tremblay and Andreas Loukas. 2020 · 2020
Cited alongside, same era.
Deep graph contrastive representation learning
Yanqiao Zhu, Yichen Xu, Feng Yu, Qiang Liu, Shu Wu, and Liang Wang. 2020 · 2020
Cited alongside, same era.
Litegt: Efficient and lightweight graph transformers. In Proceedings of the 30th ACM International Conference on Information & Knowledge Management . 161–170
Cong Chen, Chaofan Tao, and Ngai Wong. 2021 · 2021
Cited alongside, same era.
Later among the works it cites.
Influence-based mini-batching for graph neural networks. In Learning on Graphs Conference . PMLR, 9–1
Johannes Gasteiger, Chendi Qian, and Stephan Günnemann. 2022 · 2022
Later among the works it cites.
Negative sampling strategies for contrastive self-supervised learning of graph representations
Hakim Hafidi, Mounir Ghogho, Philippe Ciblat, and Ananthram Swami. 2022 · 2022
Later among the works it cites.
Masked autoencoders are scalable vision learners. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition . 16000–16009
Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li, Piotr Dollár, and Ross Girshick. 2022 · 2022
Later among the works it cites.
Sgkd: A scalable and effective knowledge distillation framework for graph representation learning. In 2022 IEEE International Conference on Data Mining Workshops (ICDMW) . IEEE, 666–673
Yufei He and Yao Ma. 2022 · 2022
Later among the works it cites.
Graphmae: Self-supervised masked graph autoencoders. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining . 594–604
Zhenyu Hou, Xiao Liu, Yukuo Cen, Yuxiao Dong, Hongxia Yang, Chunjie Wang, and Jie Tang. 2022 · 2022
Later among the works it cites.
Nodeformer: A scalable graph structure learning transformer for node classification
Qitian Wu, Wentao Zhao, Zenan Li, David P Wipf, and Junchi Yan. 2022 · 2022
Later among the works it cites.
Graph masked autoencoders with transformers
Sixiao Zhang, Hongxu Chen, Haoran Yang, Xiangguo Sun, Philip S Yu, and Guandong Xu. 2022a · 2022
Later among the works it cites.
Hierarchical graph transformer with adaptive node sampling
Zaixi Zhang, Qi Liu, Qingyong Hu, and Chee-Kong Lee. 2022b · 2022
Later among the works it cites.
When to Pre-Train Graph Neural Networks? An Answer from Data Generation Perspective!
Yuxuan Cao, Jiarong Xu, Carl Yang, Jiaan Wang, Yunchao Zhang, Chunping Wang, Lei Chen, and Yang Yang. 2023 · 2023
Later among the works it cites.
Explanations as Features: LLM-Based Features for Text-Attributed Graphs
Xiaoxin He, Xavier Bresson, Thomas Laurent, and Bryan Hooi. 2023 · 2023
Later among the works it cites.
GraphMAE2: A Decoding-Enhanced Masked Self-Supervised Graph Learner. In Proceedings of the ACM Web Conference 2023 . 737–746
Zhenyu Hou, Yufei He, Yukuo Cen, Xiao Liu, Yuxiao Dong, Evgeny Kharlamov, and Jie Tang. 2023 · 2023
Later among the works it cites.
Maskgae: Masked graph modeling meets graph autoencoders
Jintang Li, Ruofan Wu, Wangbin Sun, Liang Chen, Sheng Tian, Liang Zhu, Changhua Meng, Zibin Zheng, and Weiqiang Wang. 2023 · 2023
Later among the works it cites.
InferTurbo: A Scalable System for Boosting Full-graph Inference of Graph Neural Network over Huge Graphs. In 2023 IEEE 39th International Conference on Data Engineering (ICDE) . IEEE, 3235–3247
Dalong Zhang, Xianzheng Song, Zhiyang Hu, Yang Li, Miao Tao, Binbin Hu, Lin Wang, Zhiqiang Zhang, and Jun Zhou. 2023 · 2023
Later among the works it cites.
UniGraph: Learning a Cross-Domain Graph Foundation Model From Natural Language
Yufei He and Bryan Hooi. 2024 · 2024
Closest in time.
FiDeLiS: Faithful Reasoning in Large Language Model for Knowledge Graph Question Answering
Yuan Sui, Yufei He, Nian Liu, Xiaoxin He, Kun Wang, and Bryan Hooi. 2024 · 2024
Closest in time.
Yanqiao Zhu, Yichen Xu, Feng Yu, Qiang Liu, Shu Wu, and Liang Wang. 2021 · 2080
Closest in time.