Training Matters: Unlocking Potentials of Deeper Graph Convolutional Neural Networks

Understand

The performance limit of Graph Convolutional Networks (GCNs) and the fact that we cannot stack more of them to increase the performance, which we usually do for other deep learning paradigms, are pervasively thought to be caused by the limitations of the GCN layers, including insufficient expressive power, etc.

• However, if so, for a fixed architecture, it would be unlikely to lower the training difficulty and to improve performance by changing only the training procedure, which we show in this paper not only possible but possible in several ways.
• This paper first identify the training difficulty of GCNs from the perspective of graph signal energy loss.
• More specifically, we find that the loss of energy in the backward pass during training nullifies the learning of the layers closer to the input.