Weakly-Supervised Image Semantic Segmentation Using Graph Convolutional Networks

31 Mar 2021  ·  Shun-Yi Pan, Cheng-You Lu, Shih-Po Lee, Wen-Hsiao Peng ·

This work addresses weakly-supervised image semantic segmentation based on image-level class labels. One common approach to this task is to propagate the activation scores of Class Activation Maps (CAMs) using a random-walk mechanism in order to arrive at complete pseudo labels for training a semantic segmentation network in a fully-supervised manner. However, the feed-forward nature of the random walk imposes no regularization on the quality of the resulting complete pseudo labels. To overcome this issue, we propose a Graph Convolutional Network (GCN)-based feature propagation framework. We formulate the generation of complete pseudo labels as a semi-supervised learning task and learn a 2-layer GCN separately for every training image by back-propagating a Laplacian and an entropy regularization loss. Experimental results on the PASCAL VOC 2012 dataset confirm the superiority of our scheme to several state-of-the-art baselines. Our code is available at https://github.com/Xavier-Pan/WSGCN.

PDF Abstract
Task Dataset Model Metric Name Metric Value Global Rank Result Benchmark
Weakly-Supervised Semantic Segmentation PASCAL VOC 2012 test WSGCN (MS-COCO-pre-trained weights) Mean IoU 69.3 # 46
Weakly-Supervised Semantic Segmentation PASCAL VOC 2012 test WSGCN (no Saliency map) Mean IoU 68.8 # 48
Weakly-Supervised Semantic Segmentation PASCAL VOC 2012 val WSGCN (MS-COCO-pre-trained weights) Mean IoU 68.7 # 50
Weakly-Supervised Semantic Segmentation PASCAL VOC 2012 val WSGCN (no Saliency map) Mean IoU 66.7 # 62

Methods