2019

Graph Warp Module: an Auxiliary Module for Boosting the Power of Graph Neural Networks in Molecular Graph Analysis

Ishiguro, Katsuhiko, Maeda, Shin-ichi, Koyama, Masanori

Understand

Graph Neural Network (GNN) is a popular architecture for the analysis of chemical molecules, and it has numerous applications in material and medicinal science.

Current lines of GNNs developed for molecular analysis, however, do not fit well on the training set, and their performance does not scale well with the complexity of the network.
In this paper, we propose an auxiliary module to be attached to a GNN that can boost the representation power of the model without hindering with the original GNN architecture.
Our auxiliary module can be attached to a wide variety of GNNs, including those that are used commonly in biochemical applications.

Built on

Enumeration of 166 billion organic small molecules in the chemical universe database gdb-17
Lars Ruddigkeit, Ruud van Deursen, Lorenz C. Blum, and Reymond Jean-Louis · 2012
Earlier work this paper cites.
Spectral Networks and Deep Locally Connected Networks on Graphs
Joan Bruna and Arthur Szlam · 2014
Earlier work this paper cites.
Learning phrase representations using rnn encoder-decoder for statistical machine translation
Kyunghyun Cho, Bart Van Merriënboer, Caglar Gulcehre, Dzmitry Bahdanau, Fethi Bougares, Holger Schwenk, and Yoshua Bengio · 2014
Earlier work this paper cites.
Quantum chemistry structures and properties of 134 kilo molecules
Raghunathan Ramakrishnan, Pavlo O. Dral, Matthias Rupp, and O. Anatole von Lilienfeld · 2014
Earlier work this paper cites.
Dropout: A Simple Way of Prevent Neural Networks from Overfitting
Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov · 2014
Earlier work this paper cites.
Convolutional Networks on Graphs for Learning Molecular Fingerprints
David Duvenaud, Dougal Maclaurin, Jorge Aguilera-Iparraguirre, Rafael Gómez-Bombarelli, Timothy Hirzel, Alán Aspuru-Guzik, and Ryan P. Adams · 2015
Earlier work this paper cites.
Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
Sergey Ioffe and Christian Szegedy · 2015
Earlier work this paper cites.
Adam: a Method for Stochastic Optimization
Diederik P. Kingma and Jimmy Lei Ba · 2015
Earlier work this paper cites.
Effective Approaches to Attention-based Neural Machine Translation
Minh-Thang Luong, Hieu Pham, and Christopher D. Manning · 2015
Earlier work this paper cites.
Chainer: a Next-Generation Open Source Framework for Deep Learning
Seiya Tokui, Kenta Oono, Shohei Hido, and Justin Clayton · 2015
Earlier work this paper cites.
Convolutional Neural Networks on Graphs with Fast Localized Spectral Filtering
Michaël Defferrard, Xavier Bresson, and Pierre Vandergheynst · 2016
Earlier work this paper cites.

Similar

Deep Residual Learning for Image Recognition
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun · 2016
Cited alongside, same era.
Molecular graph convolutions: moving beyond fingerprints
Steven Kearnes, Kevin McCloskey, Marc Berndl, Vijay Pande, and Patrick Riley · 2016
Cited alongside, same era.
Gated Graph Sequence Neural Networks
Yujia Li, Daniel Tarlow, Marc Brockschmidt, and Richard Zemel · 2016
Cited alongside, same era.
Neural Message Passing for Quantum Chemistry
Justin Gilmer, Samuel S. Schoenholz, Patrick F. Riley, Oriol Vinyals, and George E. Dahl · 2017
Cited alongside, same era.
Inductive representation learning on large graphs
Will Hamilton, Zhitao Ying, and Jure Leskovec · 2017
Cited alongside, same era.
Semi-supervised Classification with Graph Convolutional Networks
Thomas N. Kipf and Max Welling · 2017
Cited alongside, same era.
Learning Graph-Level Representation for Drug Discovery
Junying Li, Deng Cai, and Xiaofei He · 2017
Cited alongside, same era.
Graph Classification via Deep Learning with Virtual Nodes
Trang Pham, Truyen Tran, Hoa Dam, and Svetha Venkatesh · 2017
Cited alongside, same era.
Modeling Relational Data with Graph Convolutional Networks
Michael Schlichtkrull, Thomas N. Kipf, Peter Bloem, Rianne van den Berg, Ivan Titov, and Max Welling · 2017
Cited alongside, same era.
Attention Is All You Need
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin · 2017
Cited alongside, same era.
Optuna: A hyperparameter optimization framework
Preferred Networks · 2018
Cited alongside, same era.

Then

Relational inductive biases , deep learning , and graph networks
Peter W. Battaglia, Jessica B. Hamrick, Victor Bapst, Alvaro Sanchez-gonzalez, Vinicius Zambaldi, Mateusz Malinowski, Andrea Tacchetti, David Raposo, Adam Santoro, Ryan Faulkner, Caglar Gulcehre, Francis Song, Andrew Ballard, Justin Gilmer, George Dahl, Ashish Vaswani, Kelsey Allen, Charles Nash, Victoria Langston, Chris Dyer, Nicolas Heess, Daan Wierstra, Pushmeet Kohli, Matt Botvinick, Oriol Vinyals, Yujia Li, and Razvan Pascanu · 2018
Later among the works it cites.
Relational Graph Attention Networks
Dan Busbridge, Dane Sherburn, Pietro Cavallo, and Nils Y. Hammerla · 2018
Later among the works it cites.
Deeper Insights into Graph Convolutional Networks for Semi-supervised Learning
Qimai Li, Zhichao Han, and Xiao-ming Wu · 2018
Later among the works it cites.
Spectral Normalization for Generative Adversarial Networks
Takeru Miyato, Toshiki Kataoka, Masanori Koyama, and Yuichi Yoshida · 2018
Later among the works it cites.
Edge Attention-based Multi-Relational Graph Convolutional Networks
Chao Shang, Qinqing Liu, Ko-shin Chen, Jiangwen Sun, Jin Lu, Jinfeng Yi, and Jinbo Bi · 2018
Later among the works it cites.
Graph Attention Networks
Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Liò, and Yoshua Bengio · 2018
Later among the works it cites.
Non-local Neural Networks
Xiaolong Wang, Ross Girshick, Abhinav Gupta, and Kaiming He · 2018
Later among the works it cites.
MoleculeNet : A Benchmark for Molecular Machine Learning
Zhenqin Wu, Bharath Ramsundar, Evan N Feinberg, Joseph Gomes, Caleb Geniesse, Aneesh S Pappu, Karl Leswing, and Vijay Pande · 2018
Later among the works it cites.
Hierarchical Graph Representation Learning with Differentiable Pooling
Rex Ying, Jiaxuan You, Christopher Morris, Xiang Ren, William L. Hamilton, and Jure Leskovec · 2018
Later among the works it cites.
How Powerful are Graph Neural Networks?
Keyulu Xu, Weihua Hu, Jure Leskovec, and Stefanie Jegelka · 2019
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…