2020

EdgeNets:Edge Varying Graph Neural Networks

Isufi, Elvin, Gama, Fernando, Ribeiro, Alejandro

Understand

Driven by the outstanding performance of neural networks in the structured Euclidean domain, recent years have seen a surge of interest in developing neural networks for graphs and data supported on graphs.

The graph is leveraged at each layer of the neural network as a parameterization to capture detail at the node level with a reduced number of parameters and computational complexity.
Following this rationale, this paper puts forth a general framework that unifies state-of-the-art graph neural networks (GNNs) through the concept of EdgeNet.
An EdgeNet is a GNN architecture that allows different nodes to use different parameters to weigh the information of different neighbors.

Built on

Z. Wu, S. Pan, F. Chen, G. Long, C. Zhang, and P. S. Yu, “A comprehensive survey on graph neural networks,” arXiv:1901.00596v3 [cs.LG] , 8 Aug. 2019. [Online]. Available: http://arxiv.org/abs/1901.00596
Original
1901
Earlier work this paper cites.
F. M. Bianchi, D. Grattarola, C. Alippi, and L. Livi, “Graph neural networks with convolutional ARMA filters,” arXiv:1901.01343v5 [cs.LG] , 24 Oct. 2019. [Online]. Available: http://arxiv.org/abs/1901.01345
Original
1901
Earlier work this paper cites.
G. Taubin, “Geometric signal processing on polygonal meshes,” in EUROGRAPHICS ’2000 . Interlaken, Switzerland: The Eurographics Association, 21-25 Aug. 2000, pp. 1–11
2000
Earlier work this paper cites.
N. A. Heckert and J. J. Filliben, “Nist/sematech e-handbook of statistical methods; chapter 1: Exploratory data analysis,” 2003
2003
Earlier work this paper cites.
F. Scarselli, S. L. Yong, M. Gori, M. Hagenbuchner, A. C. Tso, and M. Maggini, “Graph neural networks for ranking web pages,” in The 2005 IEEE/WIC/ACM Int. Conf. Web Intelligence . Compiegne, France: IEEE, 19-22 Sep. 2005, pp. 1–7
2005
Earlier work this paper cites.
N. Wale, I. A. Watson, and G. Karypis, “Comparison of descriptor spaces for chemical compound retrieval and classification,” Knowledge and Information Systems , vol. 14, no. 3, pp. 347–375, 2008
2008
Earlier work this paper cites.
S. Liu and G. Trenkler, “Hadamard, khatri-rao, kronecker and other matrix products,” International Journal of Information and Systems Sciences , vol. 4, no. 1, pp. 160–177, 2008
2008
Earlier work this paper cites.
L. Tang and H. Liu, “Relational learning via latent social dimensions,” in 15th ACM SIGKDD Int. Conf. Knowledge Discovery and Data Mining . Paris, France: ACM, 28 June-1 July 2009, pp. 817–826
2009
Earlier work this paper cites.
F. Scarselli, M. Gori, A. C. Tsoi, M. Hagenbuchner, and G. Monfardini, “The graph neural network model,” IEEE Trans. Neural Netw. , vol. 20, no. 1, pp. 61–80, Jan. 2009
2009
Earlier work this paper cites.
K. Gregor and Y. LeCun, “Learning fast approximations of sparse coding,” in Proceedings of the 27th international conference on international conference on machine learning , 2010, pp. 399–406
2010
Earlier work this paper cites.
J. McAuley and J. Leskovec, “Learning to discover social circles in Ego networks,” in 26th Conf. Neural Inform. Process. Syst. Stateline, TX: Neural Inform. Process. Foundation, 3-8 Dec. 2012
2012
Earlier work this paper cites.
D. I. Shuman, S. K. Narang, P. Frossard, A. Ortega, and P. Vandergheynst, “The emerging field of signal processing on graphs: Extending high-dimensional data analysis to networks and other irregular domains,” IEEE Signal Process. Mag. , vol. 30, no. 3, pp. 83–98, May 2013
2013
Earlier work this paper cites.
A. Sandryhaila and J. M. F. Moura, “Discrete signal processing on graphs,” IEEE Trans. Signal Process. , vol. 61, no. 7, pp. 1644–1656, Apr. 2013
2013
Earlier work this paper cites.
S. K. Narang, “Compact support biorthogonal wavelet filterbanks for arbitrary undirected graphs,” IEEE Trans. Signal Process. , vol. 61, no. 19, pp. 4673–4685, Oct. 2013
2013
Earlier work this paper cites.
A. Banerjee, A. G. Chandrasekhar, E. Duflo, and M. O. Jackson, “The diffusion of microfinance,” Science , vol. 341, no. 6144, pp. 1 236 498 (1–7), July 2013
2013
Earlier work this paper cites.
J. Bruna, W. Zaremba, A. Szlam, and Y. LeCun, “Spectral networks and deep locally connected networks on graphs,” in 2nd Int. Conf. Learning Representations . Banff, AB: Assoc. Comput. Linguistics, 14-16 Apr. 2014, pp. 1–14
2014
Earlier work this paper cites.
A. Sandyhaila and J. M. F. Moura, “Discrete signal processing on graphs: Frequency analysis,” IEEE Trans. Signal Process. , vol. 62, no. 12, pp. 3042–3054, June 2014
2014
Earlier work this paper cites.
D. Bahdanau, K. Cho, and Y. Bengio, “Neural machine translation by jointly learning to align and translate,” in 3rd Int. Conf. Learning Representations . San Diego, CA: Assoc. Comput. Linguistics, 7-9 May 2015, pp. 1–15
2015
Earlier work this paper cites.
D. P. Kingma and J. L. Ba, “ADAM: A method for stochastic optimization,” in 3rd Int. Conf. Learning Representations . San Diego, CA: Assoc. Comput. Linguistics, 7-9 May 2015, pp. 1–15
2015
Earlier work this paper cites.
S. Segarra, M. Eisen, and A. Ribeiro, “Authorship attribution through function word adjacency networks,” IEEE Trans. Signal Process. , vol. 63, no. 20, pp. 5464–5478, Oct. 2015
2015
Earlier work this paper cites.
F. M. Harper and J. A. Konstan, “The movielens datasets: History and context,” Acm transactions on interactive intelligent systems (tiis) , vol. 5, no. 4, pp. 1–19, 2015
2015
Earlier work this paper cites.
M. Defferrard, X. Bresson, and P. Vandergheynst, “Convolutional neural networks on graphs with fast localized spectral filtering,” in 30th Conf. Neural Inform. Process. Syst. Barcelona, Spain: Neural Inform. Process. Foundation, 5-10 Dec. 2016, pp. 3844–3858
2016
Earlier work this paper cites.

Similar

J. Atwood and D. Towsley, “Diffusion-convolutional neural networks,” in 30th Conf. Neural Inform. Process. Syst. Barcelona, Spain: Neural Inform. Process. Foundation, 5-10 Dec. 2016
2016
Cited alongside, same era.
A. Anis, A. Gadde, and A. Ortega, “Efficient sampling set selection for bandlimited graph signals using graph spectral proxies,” IEEE Trans. Signal Process. , vol. 64, no. 14, pp. 3775–3789, July 2016
2016
Cited alongside, same era.
M. M. Bronstein, J. Bruna, Y. LeCun, A. Szlam, and P. Vandergheynst, “Geometric deep learning: Going beyond Euclidean data,” IEEE Signal Process. Mag. , vol. 34, no. 4, pp. 18–42, July 2017
2017
Cited alongside, same era.
T. N. Kipf and M. Welling, “Semi-supervised classification with graph convolutional networks,” in 5th Int. Conf. Learning Representations . Toulon, France: Assoc. Comput. Linguistics, 24-26 Apr. 2017, pp. 1–14
2017
Cited alongside, same era.
M. Simonovsky and N. Komodakis, “Dynamic edge-conditioned filters in convolutional neural networks on graphs,” in Conf. Comput. Vision and Pattern Recognition 2017 . Honolulu, HI: Comput. Vision Foundation, 21-26 July 2017, pp. 3693–3702
2017
Cited alongside, same era.
F. Monti, D. Boscaini, J. Masci, E. Rodolà, J. Svoboda, and M. M. Bronstein, “Geometric deep learning on graphs and manifolds using mixture model CNNs,” in Conf. Comput. Vision and Pattern Recognition 2017 . Honolulu, HI: Comput. Vision Foundation, 21-26 July 2017, pp. 3693–3702
2017
Cited alongside, same era.
O. Teke and P. P. Vaidyanathan, “Extending classical multirate signal processing theory to graphs—Part I: Fundamentals,” IEEE Trans. Signal Process. , vol. 65, no. 2, pp. 409–422, Jan. 2017
2017
Cited alongside, same era.
S. Segarra, A. G. Marques, and A. Ribeiro, “Optimal graph-filter design and applications to distributed linear network operators,” IEEE Trans. Signal Process. , vol. 65, no. 15, pp. 4117–4131, Aug. 2017
2017
Cited alongside, same era.
E. Isufi, A. Loukas, A. Simonetto, and G. Leus, “Autoregressive moving average graph filtering,” IEEE Trans. Signal Process. , vol. 65, no. 2, pp. 274–288, Jan. 2017
2017
Cited alongside, same era.
A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, “Attention is all you need,” in 31st Conf. Neural Inform. Process. Syst. Long Beach, CA: Neural Inform. Process. Syst. Foundation, 4-9 Dec. 2017, pp. 1–11
2017
Cited alongside, same era.
J. Gilmer, S. S. Schoenholz, P. F. Riley, O. Vinyals, and G. E. Dahl, “Neural message passing for quantum chemistry,” arXiv preprint arXiv:1704.01212 , 2017
Original
2017
Cited alongside, same era.
W. Hamilton, Z. Ying, and J. Leskovec, “Inductive representation learning on large graphs,” in Advances in neural information processing systems , 2017, pp. 1024–1034
2017
Cited alongside, same era.

Then

H. Maron, H. Ben-Hamu, N. Shamir, and Y. Lipman, “Invariant and equivariant graph networks,” arXiv preprint arXiv:1812.09902 , 2018
Original
2018
Later among the works it cites.
F. Gama, G. Leus, A. G. Marques, and A. Ribeiro, “Convolutional neural networks via node-varying graph filters,” in 2018 IEEE Data Sci. Workshop . Lausanne, Switzerland: IEEE, 4-6 June 2018, pp. 220–224
2018
Later among the works it cites.
E. Isufi, F. Gama, and A. Ribeiro, “Generalizing graph convolutional neural networks with edge-variant recursions on graphs,” in 27th Eur. Signal Process. Conf. A Coruña, Spain: Eur. Assoc. Signal Process., 2-6 Sep. 2019
2019
Later among the works it cites.
F. Gama, A. G. Marques, G. Leus, and A. Ribeiro, “Convolutional neural network architectures for signals supported on graphs,” IEEE Trans. Signal Process. , vol. 67, no. 4, pp. 1034–1049, Feb. 2019
2019
Later among the works it cites.
K. Xu, W. Hu, J. Leskovec, and S. Jegelka, “How powerful are graph neural networks?” in 7th Int. Conf. Learning Representations . New Orleans, LA: Assoc. Comput. Linguistics, 6-9 May 2019, pp. 1–17
2019
Later among the works it cites.
R. Levie, F. Monti, X. Bresson, and M. M. Bronstein, “CayleyNets: Graph convolutional neural networks with complex rational spectral filters,” IEEE Trans. Signal Process. , vol. 67, no. 1, pp. 97–109, Jan. 2019
2019
Later among the works it cites.
J. Zhou, G. Cui, Z. Zhang, C. Yang, Z. Liu, L. Wang, C. Li, and M. Sun, “Graph neural networks: A review of methods and applications,” arXiv:1812.08434v4 [cs.LG] , 10 July 2019. [Online]. Available: http://arxiv.org/abs/1812.08434
Original
2019
Later among the works it cites.
M. Coutino, E. Isufi, and G. Leus, “Advances in distributed graph filtering,” IEEE Trans. Signal Process. , vol. 67, no. 9, pp. 2320–2333, May 2019
2019
Later among the works it cites.
V. N. Ioannidis, A. G. Marques, and G. B. Giannakis, “A recurrent graph neural network for multi-relational data,” in 44th IEEE Int. Conf. Acoust., Speech and Signal Process. Brighton, UK: IEEE, 12-17 May 2019, pp. 8157–8161
2019
Later among the works it cites.
R. Levie, E. Isufi, and G. Kutyniok, “On the transferability of spectral graph filters,” in 13th Int. Conf. Sampling Theory Applications . Bordeaux, France: IEEE, 8-12 July 2019, pp. 1–5
2019
Later among the works it cites.
L. N. Trefethen, Approximation Theory and Approximation Practice, Extended Edition . SIAM, 2019
2019
Later among the works it cites.
J. Liu, E. Isufi, and G. Leus, “Filter design for autoregressive moving average graph filters,” IEEE Trans. Signal, Inform. Process. Networks , vol. 5, no. 1, pp. 47–60, March 2019
2019
Later among the works it cites.
W. A. S. Wijesinghe and Q. Wang, “Dfnets: Spectral cnns for graphs with feedback-looped filters,” in Advances in Neural Information Processing Systems , 2019, pp. 6007–6018
2019
Later among the works it cites.
L. Ruiz, F. Gama, A. G. Marques, and A. Ribeiro, “Invariance-preserving localized activation functions for graph neural networks,” IEEE Trans. Signal Process. , vol. 68, pp. 127–141, 25 Nov. 2019
2019
Later among the works it cites.
H. Maron, H. Ben-Hamu, H. Serviansky, and Y. Lipman, “Provably powerful graph networks,” in Advances in neural information processing systems , 2019, pp. 2156–2167
2019
Later among the works it cites.
C. Morris, M. Ritzert, M. Fey, W. L. Hamilton, J. E. Lenssen, G. Rattan, and M. Grohe, “Weisfeiler and leman go neural: Higher-order graph neural networks,” in Proceedings of the AAAI Conference on Artificial Intelligence , vol. 33, 2019, pp. 4602–4609
2019
Later among the works it cites.
R. Levie, W. Huang, L. Bucci, M. M. Bronstein, and G. Kutyniok, “Transferability of spectral graph convolutional neural networks,” arXiv preprint arXiv:1907.12972 , 2019
Original
2019
Later among the works it cites.
F. Gama, E. Isufi, G. Leus, and A. Ribeiro, “Graphs, convolutions, and neural networks: From graph filters to graph neural networks,” IEEE Signal Process. Mag. , vol. 37, no. 6, pp. 128–138, Nov. 2020
2020
Closest in time.
J. You, J. Leskovec, K. He, and S. Xie, “Graph structure of neural networks,” in International Conference on Machine Learning . PMLR, 2020, pp. 10 881–10 891
2020
Closest in time.
F. Gama, J. Bruna, and A. Ribeiro, “Stability properties of graph neural networks,” IEEE Trans. Signal Process. , vol. 68, pp. 5680–5695, 25 Sep. 2020
2020
Closest in time.
L. Ruiz, F. Gama, and A. Ribeiro, “Gated graph recurrent neural networks,” IEEE Trans. Signal Process. , vol. 68, pp. 6303–6318, 26 Oct. 2020
2020
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…