Fetching the paper…

Reading the bibliography…

2023

PSP: Pre-Training and Structure Prompt Tuning for Graph Neural Networks

Ge, Qingqing, Zhao, Zeyuan, Liu, Yiding et al.

Original
PDF
Post on X

Understand

Graph Neural Networks (GNNs) are powerful in learning semantics of graph data.

Recently, a new paradigm "pre-train and prompt" has shown promising results in adapting GNNs to various tasks with less supervised data.
The success of such paradigm can be attributed to the more consistent objectives of pre-training and task-oriented prompt tuning, where the pre-trained knowledge can be effectively transferred to downstream tasks.
Most existing methods are based on the class prototype vector framework.

PSP: Pre-Training and Structure Prompt Tuning for Graph Neural Networks · Around

Built on

Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014)
Original
2014
Earlier work this paper cites.
Kipf, T.N., Welling, M.: Semi-supervised classification with graph convolutional networks. arXiv preprint arXiv:1609.02907 (2016)
Original
2016
Earlier work this paper cites.
Hamilton, W., Ying, Z., Leskovec, J.: Inductive representation learning on large graphs. NeurIPS 30
2017
Earlier work this paper cites.
Veličković, P., Cucurull, G., Casanova, A., Romero, A., Lio, P., Bengio, Y.: Graph attention networks. arXiv preprint arXiv:1710.10903 (2017)
Original
2017
Earlier work this paper cites.
Veličković, P., Fedus, W., Hamilton, W.L., Liò, P., Bengio, Y., Hjelm, R.D.: Deep graph infomax. arXiv preprint arXiv:1809.10341 (2018)
Original
2018
Earlier work this paper cites.
Wang, D., Lin, J., Cui, P., Jia, Q., Wang, Z., Fang, Y., Yu, Q., Zhou, J., Yang, S., Qi, Y.: A semi-supervised graph attentive network for financial fraud detection. In: ICDM. pp. 598–607. IEEE (2019)
2019
Earlier work this paper cites.
Chen, T., Kornblith, S., Norouzi, M., Hinton, G.: A simple framework for contrastive learning of visual representations. In: International conference on machine learning. pp. 1597–1607. PMLR (2020)
2020
Earlier work this paper cites.
Hu, W., Fey, M., Zitnik, M., Dong, Y., Ren, H., Liu, B., Catasta, M., Leskovec, J.: Open graph benchmark: Datasets for machine learning on graphs. NeurIPS 33
2020
Earlier work this paper cites.
Hu, Z., Dong, Y., Wang, K., Chang, K.W., Sun, Y.: Gpt-gnn: Generative pre-training of graph neural networks. In: KDD. pp. 1857–1867 (2020)
2020
Earlier work this paper cites.
Jin, W., Derr, T., Liu, H., Wang, Y., Wang, S., Liu, Z., Tang, J.: Self-supervised learning on graphs: Deep insights and new direction. arXiv preprint arXiv:2006.10141 (2020)
Original
2020
Earlier work this paper cites.
Morris, C., Kriege, N.M., Bause, F., Kersting, K., Mutzel, P., Neumann, M.: Tudataset: A collection of benchmark datasets for learning with graphs. arXiv preprint arXiv:2007.08663 (2020)
Original
2020
Earlier work this paper cites.

Similar

Pei, H., Wei, B., Chang, K.C.C., Lei, Y., Yang, B.: Geom-gcn: Geometric graph convolutional networks. arXiv preprint arXiv:2002.05287 (2020)
Original
2020
Cited alongside, same era.
Qiu, X., Sun, T., Xu, Y., Shao, Y., Dai, N., Huang, X.: Pre-trained models for natural language processing: A survey. Science China Technological Sciences 63
2020
Cited alongside, same era.
You, Y., Chen, T., Sui, Y., Chen, T., Wang, Z., Shen, Y.: Graph contrastive learning with augmentations. NeurIPS 33
2020
Cited alongside, same era.
Lim, D., Hohne, F., Li, X., Huang, S.L., Gupta, V., Bhalerao, O., Lim, S.N.: Large scale learning on non-homophilous graphs: New benchmarks and strong simple methods. NeurIPS 34
2021
Cited alongside, same era.
Lu, Y., Jiang, X., Fang, Y., Shi, C.: Learning to pre-train graph neural networks. In: AAAI. vol. 35, pp. 4276–4284 (2021)

Then

Sun, M., Zhou, K., He, X., Wang, Y., Wang, X.: Gppt: Graph pre-training and prompt tuning to generalize graph neural networks. In: KDD. pp. 1717–1727 (2022)
2022
Later among the works it cites.
Wu, S., Sun, F., Zhang, W., Xie, X., Cui, B.: Graph neural networks in recommender systems: a survey. ACM Computing Surveys 55
2022
Later among the works it cites.
Xia, J., Zhu, Y., Du, Y., Li, S.Z.: A survey of pretraining on graphs: Taxonomy, methods, and applications. arXiv preprint arXiv:2202.07893 (2022)
Original
2022
Later among the works it cites.
Chen, M., Liu, Z., Liu, C., Li, J., Mao, Q., Sun, J.: Ultra-dp: Unifying graph pre-training with multi-task graph dual prompt. arXiv preprint arXiv:2310.14845 (2023)
Original
2023
Closest in time.
Li, X., Ye, T., Shan, C., Li, D., Gao, M.: Seegera: Self-supervised semi-implicit graph variational auto-encoders with masking. In: WebConf. pp. 143–153 (2023)

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

2021

Cited alongside, same era.

Wan, S., Zhan, Y., Liu, L., Yu, B., Pan, S., Gong, C.: Contrastive graph poisson networks: Semi-supervised learning with extremely limited labels. NeurIPS 34

2021

Cited alongside, same era.

Zhu, Y., Xu, Y., Yu, F., Liu, Q., Wu, S., Wang, L.: Graph contrastive learning with adaptive augmentation. In: WebConf. pp. 2069–2080 (2021)

2021

Cited alongside, same era.

Ding, K., Wang, J., Caverlee, J., Liu, H.: Meta propagation networks for graph few-shot semi-supervised learning. In: AAAI. vol. 36, pp. 6524–6531 (2022)

2022

Cited alongside, same era.

Fang, T., Zhang, Y., Yang, Y., Wang, C., Chen, L.: Universal prompt tuning for graph neural networks. arXiv preprint arXiv:2209.15240 (2022)

2022

Cited alongside, same era.

Hou, Z., Liu, X., Cen, Y., Dong, Y., Yang, H., Wang, C., Tang, J.: Graphmae: Self-supervised masked graph autoencoders. In: KDD. pp. 594–604 (2022)

2022

Cited alongside, same era.

Long, S., Cao, F., Han, S.C., Yang, H.: Vision-and-language pretrained models: A survey. arXiv preprint arXiv:2204.07356 (2022)

2022

Cited alongside, same era.

2023

Closest in time.

Liu, Z., Yu, X., Fang, Y., Zhang, X.: Graphprompt: Unifying pre-training and downstream tasks for graph neural networks. In: WebConf. pp. 417–428 (2023)

2023

Closest in time.

Sun, X., Cheng, H., Li, J., Liu, B., Guan, J.: All in one: Multi-task prompting for graph neural networks (2023)

2023

Closest in time.

Tan, Z., Guo, R., Ding, K., Liu, H.: Virtual node tuning for few-shot node classification. arXiv preprint arXiv:2306.06063 (2023)

2023

Closest in time.

Yu, X., Liu, Z., Fang, Y., Liu, Z., Chen, S., Zhang, X.: Generalized graph prompt: Toward a unification of pre-training and downstream tasks on graphs. arXiv preprint arXiv:2311.15317 (2023)

2023

Closest in time.

Yu, X., Zhou, C., Fang, Y., Zhang, X.: Multigprompt for multi-task pre-training and prompting on graphs. arXiv preprint arXiv:2312.03731 (2023)

2023

Closest in time.

Zhu, Y., Guo, J., Tang, S.: Sgl-pt: A strong graph learner with graph prompt tuning. arXiv preprint arXiv:2302.12449 (2023)

2023

Closest in time.