H²O: Heatmap by Hierarchical Occlusion

The rise of Deep Learning (DL) has led to a breakthrough in the research field of content-based multimedia indexing. Newly de- veloped systems based on complex models outperform classic ma- chine learning algorithms in object detection, image segmentation or classification tasks. However, despite their high performance, these systems still make mistakes. To be used in industrial condi- tions, these systems must be able to provide trustworthy decisions with guarantees or justifications. Therefore, it is crucial to provide means to analyze and comprehend the decision process that leads a model to its decision. Image classification implies tracking and understanding which input features the model relies on to make its prediction. This paper focuses on features attribution techniques and proposes Heatmaps by Hierarchical Occlusion (H²O), a novel method for detecting pattern-relevant features in an image. We also propose two new pairs of metrics that overcome some evaluation issues: (a) Insertion and Deletion Spearman correlation coefficients which both estimate a correlation between the computed scores in a saliency map and the importance for the model of the associated pixels in the image. (b) Insertion Positive and Deletion Negative Gradient Sums both estimate the coherence of the scores in the saliency maps. Both visual inspection and evaluation on 7 metrics show that H²O is competitive against state-of-the-art methods.