GResNet: Graph Residual Network for Reviving Deep GNNs from Suspended Animation

12 Sep 2019  ·  Jiawei Zhang, Lin Meng ·

The existing graph neural networks (GNNs) based on the spectral graph convolutional operator have been criticized for its performance degradation, which is especially common for the models with deep architectures. In this paper, we further identify the suspended animation problem with the existing GNNs. Such a problem happens when the model depth reaches the suspended animation limit, and the model will not respond to the training data any more and become not learnable. Analysis about the causes of the suspended animation problem with existing GNNs will be provided in this paper, whereas several other peripheral factors that will impact the problem will be reported as well. To resolve the problem, we introduce the GResNet (Graph Residual Network) framework in this paper, which creates extensively connected highways to involve nodes' raw features or intermediate representations throughout the graph for all the model layers. Different from the other learning settings, the extensive connections in the graph data will render the existing simple residual learning methods fail to work. We prove the effectiveness of the introduced new graph residual terms from the norm preservation perspective, which will help avoid dramatic changes to the node's representations between sequential layers. Detailed studies about the GResNet framework for many existing GNNs, including GCN, GAT and LoopyNet, will be reported in the paper with extensive empirical experiments on real-world benchmark datasets.

PDF Abstract

Results from the Paper


Task Dataset Model Metric Name Metric Value Global Rank Result Benchmark
Node Classification Citeseer LoopyNet Accuracy 71.6% # 54
Node Classification Citeseer GResNet(LoopyNet) Accuracy 73.7% # 31
Node Classification Citeseer GResNet(GAT) Accuracy 73.5% # 33
Node Classification Citeseer GResNet(GCN) Accuracy 72.7% # 41
Node Classification Cora GResNet(GCN) Accuracy 84.3% # 31
Node Classification Cora LoopyNet Accuracy 82.6% # 50
Node Classification Cora GResNet(LoopyNet) Accuracy 83.9% # 36
Node Classification Cora GResNet(GAT) Accuracy 85.5% # 23
Node Classification Pubmed LoopyNet Accuracy 81.2% # 30
Node Classification Pubmed GResNet(LoopyNet) Accuracy 83.0% # 24
Node Classification Pubmed GResNet(GAT) Accuracy 82.2% # 26
Node Classification Pubmed GResNet(GCN) Accuracy 81.7% # 29

Methods