2019

Batch Virtual Adversarial Training for Graph Convolutional Networks

Deng, Zhijie, Dong, Yinpeng, Zhu, Jun

Understand

We present batch virtual adversarial training (BVAT), a novel regularization method for graph convolutional networks (GCNs).

BVAT addresses the shortcoming of GCNs that do not consider the smoothness of the model's output distribution against local perturbations around the input.
We propose two algorithms, sample-based BVAT and optimization-based BVAT, which are suitable to promote the smoothness of the model for graph-structured data by either finding virtual adversarial perturbations for a subset of nodes far from each other or generating virtual adversarial perturbations for all nodes with an optimization process.
Extensive experiments on three citation network datasets Cora, Citeseer and Pubmed and a knowledge graph dataset Nell validate the effectiveness of the proposed method, which establishes state-of-the-art results in the semi-supervised node classification tasks.

Built on

Semi-supervised learning using gaussian fields and harmonic functions
Zhu, X., Ghahramani, Z., and Lafferty, J. D · 2003
Earlier work this paper cites.
Semi-supervised classification by low density separation
Chapelle, O. and Zien, A · 2005
Earlier work this paper cites.
Semi-supervised learning by entropy minimization
Grandvalet, Y. and Bengio, Y · 2005
Earlier work this paper cites.
Manifold regularization: A geometric framework for learning from labeled and unlabeled examples
Belkin, M., Niyogi, P., and Sindhwani, V · 2006
Earlier work this paper cites.
Collective classification in network data
Sen, P., Namata, G., Bilgic, M., Getoor, L., Galligher, B., and Eliassi-Rad, T · 2008
Earlier work this paper cites.
Deep learning via semi-supervised embedding
Weston, J., Ratle, F., Mobahi, H., and Collobert, R · 2012
Earlier work this paper cites.
Dropout training as adaptive regularization
Wager, S., Wang, S., and Liang, P. S · 2013
Earlier work this paper cites.
Adam: A method for stochastic optimization
Kingma, D. P. and Ba, J · 2014
Earlier work this paper cites.
Deepwalk: Online learning of social representations
Perozzi, B., Al-Rfou, R., and Skiena, S · 2014
Earlier work this paper cites.
Explaining and harnessing adversarial examples
Goodfellow, I. J., Shlens, J., and Szegedy, C · 2015
Earlier work this paper cites.

Similar

Line: Large-scale information network embedding
Tang, J., Qu, M., Wang, M., Zhang, M., Yan, J., and Mei, Q · 2015
Cited alongside, same era.
Convolutional neural networks on graphs with fast localized spectral filtering
Defferrard, M., Bresson, X., and Vandergheynst, P · 2016
Cited alongside, same era.
node2vec: Scalable feature learning for networks
Grover, A. and Leskovec, J · 2016
Cited alongside, same era.
Regularization with stochastic transformations and perturbations for deep semi-supervised learning
Sajjadi, M., Javanmardi, M., and Tasdizen, T · 2016
Cited alongside, same era.
Revisiting semi-supervised learning with graph embeddings
Yang, Z., Cohen, W., and Salakhudinov, R · 2016
Cited alongside, same era.
Towards evaluating the robustness of neural networks
Carlini, N. and Wagner, D · 2017
Cited alongside, same era.
Stochastic training of graph convolutional networks
Original
Chen, J. and Zhu, J · 2017
Cited alongside, same era.
Inductive representation learning on large graphs
Hamilton, W., Ying, Z., and Leskovec, J · 2017
Cited alongside, same era.
Semi-supervised classification with graph convolutional networks
Kipf, T. N. and Welling, M · 2017
Cited alongside, same era.
Temporal ensembling for semi-supervised learning
Laine, S. and Aila, T · 2017
Cited alongside, same era.

Then

Adversarial training methods for semi-supervised text classification
Miyato, T., Dai, A. M., and Goodfellow, I · 2017
Later among the works it cites.
Geometric deep learning on graphs and manifolds using mixture model cnns
Monti, F., Boscaini, D., Masci, J., Rodola, E., Svoboda, J., and Bronstein, M. M · 2017
Later among the works it cites.
Lower bounds on the robustness to adversarial perturbations
Peck, J., Roels, J., Goossens, B., and Saeys, Y · 2017
Later among the works it cites.
Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning results
Tarvainen, A. and Valpola, H · 2017
Later among the works it cites.
Adversarial attack on graph structured data
Original
Dai, H., Li, H., Tian, T., Huang, X., Wang, L., Zhu, J., and Song, L · 2018
Later among the works it cites.
Graph partition neural networks for semi-supervised classification
Original
Liao, R., Brockschmidt, M., Tarlow, D., Gaunt, A. L., Urtasun, R., and Zemel, R · 2018
Later among the works it cites.
Smooth neighbors on teacher graphs for semi-supervised learning
Luo, Y., Zhu, J., Li, M., Ren, Y., and Zhang, B · 2018
Later among the works it cites.
Virtual adversarial training: a regularization method for supervised and semi-supervised learning
Miyato, T., Maeda, S.-i., Ishii, S., and Koyama, M · 2018
Later among the works it cites.
Graph Attention Networks
Veličković, P., Cucurull, G., Casanova, A., Romero, A., Liò, P., and Bengio, Y · 2018
Later among the works it cites.
Adversarial attacks on neural networks for graph data
Zügner, D., Akbarnejad, A., and Günnemann, S · 2018
Later among the works it cites.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…