Research On Interpretability Of Deep Convolutional Networks For Image Classification Based On Gradient Localization

In recent years,deep convolutional neural networks have significantly succeeded in the fields of image classification,target detection and so on.However,it is often regarded as a black box model due to the data-driven characteristic of neural networks.Class activation mapping(CAM)uses the weighted combination of feature maps to localize the class-specific discriminative regions in the image,which provides a visual explanation for the classification results of convolutional neural networks.The local interpretation methods including CAM provides a new idea for the research of classification mechanism of convolutional neural networks.Based on the idea of CAM,an Element-wise Class Activation Mapping(ECAM)method is proposed in this paper.ECAM utilizes the element-wise gradient information to get the class activation maps,which provides a explanation for the relationship between the classification result and the class-specific discriminative regions.We illustrate that ECAM is actually a generalization of the original CAM method,which breaks the limitation of model structure used in CAM and is suitable for the deep learning models based on convolutional networks.We further evaluate the localization performance of ECAM on the ImageNet dataset,and the result turn out that ECAM achieves higher localization accuracy than the Gradient-Weighted Class Activation Mapping(Grad-CAM)method in the task of weak-supervised learning localization.In addition,ECAM also achieves good localization performance in the task of multi-target image classification,which can explain the classification results of convolutional neural networks from the perspective of category features.In the field of security of artificial intelligence,the discovery of adversarial examples threatens the application of convolutional neural networks in many fields.As far as we know,most of the methods for interpreting convolutional neural networks are proposed for real examples,but there is a lack of the interpretable methods for the adversarial examples.As a result,we apply the proposed ECAM method to the adversarial examples and it is found that the class-specific discriminative regions of the adversarial examples shift compared with those of the real examples.By the adversarial attack,the feature information of the real class discriminative region is suppressed,while the information of the target class discriminative region is expressed.Furthermore,we study the relationship between the class-specific discriminative region and the adversarial disturbance region,which may provide a breakthrough point for the future research on the identification and defense of adversarial examples.