Opening The Black Box:Deep Convolutional Neural Networks With Attention Mechanism For Aerial Scene Classification

Deep learning is nowadays progressing significantly and achieves dramatic success in a wide range of fields.In remote sensing commu-nity,convolutional neural network(CNN),as one important branch of deep learning family,shows remarkable performance in solving many classification tasks,e.g.,aerial scene classification.However,although these networks have proven to be very accurate,they have widely been regarded as "black boxes" due to their complexity,and it is unfavorable in remote sensing community.Within the framework of this master thesis,"the black boxes"have been explored by visualizing their inner world.An innovative visu-alization strategy,Ll-regularized class activation map(L1-CAM),is proposed to highlight discriminative image regions,depend on which a CNN classifies images into certain categories.By visualizing these regions,it is clear that the network is not learning knowledge " ran-domly,but deliberately.For the sake of validation,L1-CAM method is compared to three benchmark visualization strategies,deconvnet,guided backpropagation and class activation mapping techniques.Fur-thermore,L1-CAM is applied to give an insight view of "knowledge"learned by intermediate layers to draw a thorough picture of the ’black boxes.The experiment demonstrates that L1-CAM is capable of localiz-ing discriminative objects or regions,which indicates regions of high interest,when CNNs perform classification.Furthermore,according to discoveries from L1-CAM of features of different levels,we proposed LAHNet,which fuses low-and high-level features for aerial scene clas-sification,and applied it successfully to weakly supervised scene local-ization.As expected,the model perform well in these missions and the outlook is summarized eventually.