2018

Hierarchical Graph Representation Learning with Differentiable Pooling

Ying, Rex, You, Jiaxuan, Morris, Christopher et al.

Understand

Recently, graph neural networks (GNNs) have revolutionized the field of graph representation learning through effectively learned node embeddings, and achieved state-of-the-art results in tasks such as node classification and link prediction.

However, current GNN methods are inherently flat and do not learn hierarchical representations of graphs---a limitation that is especially problematic for the task of graph classification, where the goal is to predict the label associated with an entire graph.
Here we propose DiffPool, a differentiable graph pooling module that can generate hierarchical representations of graphs and can be combined with various graph neural network architectures in an end-to-end fashion.
DiffPool learns a differentiable soft cluster assignment for nodes at each layer of a deep GNN, mapping nodes to a set of clusters, which then form the coarsened input for the next GNN layer.

Built on

Processing directed acyclic graphs with recursive neural networks
M. Bianchini, M. Gori, and F. Scarselli · 2001
Earlier work this paper cites.
Distinguishing enzyme structures from non-enzymes without alignments
P. D. Dobson and A. J. Doig · 2003
Earlier work this paper cites.
Shortest-path kernels on graphs
K. M. Borgwardt and H.-P. Kriegel · 2005
Earlier work this paper cites.
Protein function prediction via graph kernels
K. M. Borgwardt, C. S. Ong, S. Schönauer, S. V. N. Vishwanathan, A. J. Smola, and H.-P. Kriegel · 2005
Earlier work this paper cites.
Automatic generation of complementary descriptors with molecular graph networks
C. Merkwirth and T. Lengauer · 2005
Earlier work this paper cites.
Weighted graph cuts without eigenvectors a multilevel approach
I. S. Dhillon, Y. Guan, and B. Kulis · 2007
Earlier work this paper cites.
The graph neural network model
F. Scarselli, M. Gori, A. C. Tsoi, M. Hagenbuchner, and G. Monfardini · 2009
Earlier work this paper cites.
Efficient graphlet kernels for large graph comparison
N. Shervashidze, S. V. N. Vishwanathan, T. H. Petri, K. Mehlhorn, and K. M. Borgwardt · 2009
Earlier work this paper cites.
LIBSVM: A library for support vector machines
C.-C. Chang and C.-J. Lin · 2011
Earlier work this paper cites.
Weisfeiler-Lehman graph kernels
N. Shervashidze, P. Schweitzer, E. J. van Leeuwen, K. Mehlhorn, and K. M. Borgwardt · 2011
Earlier work this paper cites.
ImageNet classification with deep convolutional neural networks
A. Krizhevsky, I. Sutskever, and G. E. Hinton · 2012
Earlier work this paper cites.
Scalable kernels for graphs with continuous attributes
A. Feragen, N. Kasenburg, J. Petersen, M. D. Bruijne, and K. M. Borgwardt · 2013
Earlier work this paper cites.
Deep architectures and deep learning in chemoinformatics: The prediction of aqueous solubility for drug-like molecules
A. Lusci, G. Pollastri, and P. Baldi · 2013
Earlier work this paper cites.
Spectral networks and deep locally connected networks on graphs
J. Bruna, W. Zaremba, A. Szlam, and Y. LeCun · 2014
Earlier work this paper cites.

Similar

Convolutional networks on graphs for learning molecular fingerprints
D. K. Duvenaud, D. Maclaurin, J. Iparraguirre, R. Bombarell, T. Hirzel, A. Aspuru-Guzik, and R. P. Adams · 2015
Cited alongside, same era.
Batch normalization: Accelerating deep network training by reducing internal covariate shift
S. Ioffe and C. Szegedy · 2015
Cited alongside, same era.
Order matters: Sequence to sequence for sets
O. Vinyals, S. Bengio, and M. Kudlur · 2015
Cited alongside, same era.
A structural smoothing framework for robust graph comparison
P. Yanardag and S. V. N. Vishwanathan · 2015
Cited alongside, same era.
Discriminative embeddings of latent variable models for structured data
H. Dai, B. Dai, and L. Song · 2016
Cited alongside, same era.
Convolutional neural networks on graphs with fast localized spectral filtering
M. Defferrard, X. Bresson, and P. Vandergheynst · 2016
Cited alongside, same era.
Benchmark data sets for graph kernels, 2016
K. Kersting, N. M. Kriege, C. Morris, P. Mutzel, and M. Neumann · 2016
Cited alongside, same era.
On valid optimal assignment kernels and applications to graph classification
N. M. Kriege, P.-L. Giscard, and R. Wilson · 2016
Cited alongside, same era.
Gated graph sequence neural networks
Y. Li, D. Tarlow, M. Brockschmidt, and R. Zemel · 2016
Cited alongside, same era.
Learning convolutional neural networks for graphs
M. Niepert, M. Ahmed, and K. Kutzkov · 2016
Cited alongside, same era.
Geometric deep learning: Going beyond euclidean data
M. M. Bronstein, J. Bruna, Y. LeCun, A. Szlam, and P. Vandergheynst · 2017
Cited alongside, same era.
Protein interface prediction using graph convolutional networks
A. Fout, J. Byrd, B. Shariat, and A. Ben-Hur · 2017
Cited alongside, same era.

Then

Representation learning on graphs: Methods and applications
W. L. Hamilton, R. Ying, and J. Leskovec · 2017
Later among the works it cites.
Predicting organic reaction outcomes with Weisfeiler-Lehman network
W. Jin, C. W. Coley, R. Barzilay, and T. S. Jaakkola · 2017
Later among the works it cites.
Semi-supervised classification with graph convolutional networks
T. N. Kipf and M. Welling · 2017
Later among the works it cites.
Deriving neural architectures from sequence and graph kernels
T. Lei, W. Jin, R. Barzilay, and T. S. Jaakkola · 2017
Later among the works it cites.
SchNet: A continuous-filter convolutional neural network for modeling quantum interactions
K. Schütt, P. J. Kindermans, H. E. Sauceda, S. Chmiela, A. Tkatchenko, and K. R. Müller · 2017
Later among the works it cites.
Dynamic edge-conditioned filters in convolutional neural networks on graphs
M. Simonovsky and N. Komodakis · 2017
Later among the works it cites.
SplineCNN: Fast geometric deep learning with continuous B-spline kernels
M. Fey, J. E. Lenssen, F. Weichert, and H. Müller · 2018
Closest in time.
Neural relational inference for interacting systems
T. N. Kipf, E. Fetaya, K. C. Wang, M. Welling, and R. Zemel · 2018
Closest in time.
Graph partition neural networks for semi-supervised classification
R. Liao, M. Brockschmidt, D. Tarlow, A. L. Gaunt, R. Urtasun, and R. Zemel · 2018
Closest in time.
Modeling relational data with graph convolutional networks
M. Schlichtkrull, T. N. Kipf, P. Bloem, R. van den Berg, I. Titov, and M. Welling · 2018
Closest in time.
Graph attention networks
P. Veličković, G. Cucurull, A. Casanova, A. Romero, P. Liò, and Y. Bengio · 2018
Closest in time.
Graph capsule convolutional neural networks
Original
S. Verma and Z.-L. Zhang · 2018
Closest in time.
An end-to-end deep learning architecture for graph classification
M. Zhang, Z. Cui, M. Neumann, and Y. Chen · 2018
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…