2018

Pitfalls of Graph Neural Network Evaluation

Shchur, Oleksandr, Mumme, Maximilian, Bojchevski, Aleksandar et al.

Understand

Semi-supervised node classification in graphs is a fundamental problem in graph mining, and the recently proposed graph neural networks (GNNs) have achieved unparalleled results on this task.

  • Due to their massive success, GNNs have attracted a lot of attention, and many novel architectures have been put forward.
  • In this paper we show that existing evaluation strategies for GNN models have serious shortcomings.
  • We show that using the same train/validation/test splits of the same datasets, as well as making significant changes to the training procedure (e.g.

Built on

  • The elements of statistical learning

    J. Friedman, T. Hastie, and R. Tibshirani · 2001

    Earlier work this paper cites.

  • Collective classification in network data

    P. Sen, G. Namata, M. Bilgic, L. Getoor, B. Galligher, and T. Eliassi-Rad · 2008

    Earlier work this paper cites.

  • Semi-supervised learning

    O. Chapelle, B. Scholkopf, and A. Zien · 2009

    Earlier work this paper cites.

  • Understanding the difficulty of training deep feedforward neural networks

    X. Glorot and Y. Bengio · 2010

    Earlier work this paper cites.

  • Query-driven active surveying for collective classification

    G. Namata, B. London, L. Getoor, and B. Huang · 2012

    Earlier work this paper cites.

  • Adam: A method for stochastic optimization

    D. P. Kingma and J. Ba · 2015

    Earlier work this paper cites.

Similar

  • Image-based recommendations on styles and substitutes

    J. McAuley, C. Targett, Q. Shi, and A. Van Den Hengel · 2015

    Cited alongside, same era.

  • Revisiting semi-supervised learning with graph embeddings

    Z. Yang, W. W. Cohen, and R. Salakhutdinov · 2016

    Cited alongside, same era.

  • Inductive representation learning on large graphs

    W. L. Hamilton, R. Ying, and J. Leskovec · 2017

    Cited alongside, same era.

  • Semi-supervised classification with graph convolutional networks

    T. N. Kipf and M. Welling · 2017

    Cited alongside, same era.

  • Are GANs created equal? A large-scale study

    Original

    M. Lucic, K. Kurach, M. Michalski, S. Gelly, and O. Bousquet · 2017

    Cited alongside, same era.

  • Geometric deep learning on graphs and manifolds using mixture model CNNs

    F. Monti, D. Boscaini, J. Masci, E. Rodolà, J. Svoboda, and M. M. Bronstein · 2017

    Cited alongside, same era.

Then

  • Deep gaussian embedding of attributed graphs: Unsupervised inductive learning via ranking

    A. Bojchevski and S. Günnemann · 2018

    Closest in time.

  • Troubling trends in machine learning scholarship

    Original

    Z. C. Lipton and J. Steinhardt · 2018

    Closest in time.

  • On the state of the art of evaluation in neural language models

    G. Melis, C. Dyer, and P. Blunsom · 2018

    Closest in time.

  • Graph attention networks

    P. Velickovic, G. Cucurull, A. Casanova, A. Romero, P. Liò, and Y. Bengio · 2018

    Closest in time.

  • Adversarial attacks on neural networks for graph data

    D. Zügner, A. Akbarnejad, and S. Günnemann · 2018

    Closest in time.

  • Predict then propagate: Graph neural networks meet personalized PageRank

    J. Klicpera, A. Bojchevski, and S. Günnemann · 2019

    Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…