Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation

Spatial pyramid pooling module or encode-decoder structure are used in deep neural networks for semantic segmentation task. The former networks are able to encode multi-scale contextual information by probing the incoming features with filters or pooling operations at multiple rates and multiple effective fields-of-view, while the latter networks can capture sharper object boundaries by gradually recovering the spatial information. In this work, we propose to combine the advantages from both methods. Specifically, our proposed model, DeepLabv3+, extends DeepLabv3 by adding a simple yet effective decoder module to refine the segmentation results especially along object boundaries. We further explore the Xception model and apply the depthwise separable convolution to both Atrous Spatial Pyramid Pooling and decoder modules, resulting in a faster and stronger encoder-decoder network. We demonstrate the effectiveness of the proposed model on PASCAL VOC 2012 and Cityscapes datasets, achieving the test set performance of 89.0\% and 82.1\% without any post-processing. Our paper is accompanied with a publicly available reference implementation of the proposed models in Tensorflow at \url{https://github.com/tensorflow/models/tree/master/research/deeplab}.

PDF Abstract ECCV 2018 PDF ECCV 2018 Abstract

Results from the Paper


 Ranked #1 on Semantic Segmentation on PASCAL VOC 2012 test (using extra training data)

     Get a GitHub badge
Task Dataset Model Metric Name Metric Value Global Rank Uses Extra
Training Data
Result Benchmark
Lesion Segmentation Anatomical Tracings of Lesions After Stroke (ATLAS) DeepLab v3+ Dice 0.4609 # 3
IoU 0.3458 # 2
Precision 0.5831 # 3
Semantic Segmentation Cityscapes val DeepLabv3+ (Dilated-Xception-71) mIoU 79.6 # 49
Semantic Segmentation DADA-seg DeepLabV3+ (ACDC) mIoU 26.8 # 12
Semantic Segmentation DensePASS DeepLabV3+ (ResNet-101) mIoU 32.5% # 21
Semantic Segmentation EventScape DeepLabV3+ mIoU 53.65 # 6
Semantic Segmentation MCubeS DeepLabV3+ (RGB-A-D-N) mIoU 38.13% # 12
Semantic Segmentation PASCAL VOC 2012 test DeepLabv3+ (Xception-65-JFT) Mean IoU 89.0% # 1
Semantic Segmentation PASCAL VOC 2012 test DeepLabv3+ (Xception-JFT) Mean IoU 89.0% # 1
Semantic Segmentation SkyScapes-Dense DeepLabv3+ Mean IoU 38.20 # 2
Semantic Segmentation SynPASS DeepLabv3+ mIoU 29.66% # 6
Semantic Segmentation Trans10K DeepLabV3+ mIoU 68.87% # 5
GFLOPs 37.98 # 7
Semantic Segmentation UrbanLF DeepLabV3+ (ResNet-101) mIoU (Real) 76.27 # 10
mIoU (Syn) 75.39 # 12

Methods