Transfer Feature Based High-Resolution Remote Sensing Scene Classification

With the rapid development of remote sensing(RS)sensor technology and mapping technology,the quality and quantity of RS images have been greatly improved.In order to fully detect the rich knowledge contained in massive RS images and realize the information value-added of RS data,the importance of RS image interpretation is highlighted.As a basic task of RS data interpretation,high-resolution RS scene classification is an important way to understand the semantic information of RS scenes.Facing the RS scene with rich color and texture information,diverse types of ground objects and complex spatial distribution,how to effectively extract efficient expression features for scene semantic recognition is the key problem in the current RS image interpretation research.The RS scene classification based on transfer features uses the convolutional neural networks(CNN)trained by large-scale natural image dataset to express the scene information of RS image,it realizes the extraction of high-level features of RS scene with the limited training RS scene samples,and obtains the classification performance far beyond the traditional low-level image features.Although there are many similarities between natural image and RS scene in the low-level and local information,which makes the transfer features have high adaptability to express RS scene information,the domain differences between them also lead to the insufficient expression of different transfer features for RS scene information,which are mainly summarized as follows : 1)at the object level,the different object distribution is ignored,resulting in the insufficient expression of RS scene by full-connected(FC)layer features.In natural images,the objects related to image semantic are simple with single type and tend to be distributed in the central region of the image,which makes the central region have a more impact when FC layer extracts the global image information,while the surface objects in RS scene are diverse and distributed randomly,thus FC layer features from pre-trained CNNs does not fully mine the RS scene information;2)at the scene level,the content complexity difference is ignored,leading to the insufficient expression of complex RS image scene content by single convolution layer feature or FC layer feature.Compared with the natural images with single object type and simple spatial distribution,the rich ground objects types and diverse spatial distribution in the RS image scene present more complex scene content,thus single type transfer feature is hard to fully describe the global and local scene information simultaneously;3)at the dataset level,the difference of similarity between scenes is ignored,resulting in the lack of distinguishing RS scenes at different similar level by unified transfer features.Most images of these two types datasets have obvious appearance differences,however the intra-class diversity and inter-class similarity of RS scenes make the appearance differences of some scenes subtle.The unified transfer features can not express the key local details of subtle differences when distinguish the significant difference scenes.In order to solve the above problems,based on the similarities and differences between natural image and RS scenes,this paper researches pre-trained CNN feature based transfer learning for RS scene semantic understanding.Combined with the characteristic of convolution layer features and FC layer features,this paper improves the performance of transfer features by feature extraction,feature fusion and decision fusion.The main contribution and work of this paper are as follows:1.Aiming at the problem that the FC layer features can not fully express the RS scene,this paper construct a FC layer feature classification method considering the object distribution difference.An deep salient feature based anti-noise transfer network is proposed.Firstly,the salient region related to scene semantics is extracted by attention mechanism,and its key object is centered as input images of pre-trained CNN,which guides FC layer to fully mine the information of salient region to form deep salient features with strong expression ability.Then,the anti-noise transfer network is trained under the guidance of deep metric learning and noise samples,thus to improve the adaptability and robustness of RS scene features.The experimental results show that the method can effectively improve the expressive and distinguishing ability of the FC layer features for RS scene.2.Aiming at the problem that the single type transfer feature can not fully express the complex RS scene,a classification method combining convolution layer and FC layer features is proposed.Firstly,the bag-of-visual-words(Bo VW)model is used to analyze the expression ability of convolution layer features of deep convolution neural network(DCNN)VGG19,and the middle-high level convolution layer features are proved to have the best comprehensive performance in the local information expression ability and computational efficiency.Then the convolution features decoded by Bo VW is fused with the global features extracted by FC layers in the same CNN,thus to obtain fusion features considering both global and local information of RS scene,which fully express different level of information for RS scenes.Experimental results show that in this method,the transfer features' ability of distinguish complex RS scene is effectively improved.3.To solve the problem that unified transfer feature is hard to distinguish RS scenes with different similarity,a decision fusion method based on two-channel features transfer is proposed,which presents a transfer learning based mixture-of-expert classification model.The global and local features of RS scene are used to distinguish the obvious and subtle differences between scenes by the pre-judged channel and the expert channel successively.The expert channel establishes an expert transfer network for each RS scene class to mine the key local information of RS scenes from corresponding class,thus obtaining more discriminative local features for RS scenes to detect their subtle differences.Combining the decision of these two channel can discover the multi-level key information embedded in different transfer features.Experimental results show that the proposed method has higher discrimination ability than the classification method based on unified transfer feature.In this paper,CNN features trained on large natural image dataset are applied to the classification of high-resolution RS scene by transfer learning.According to the characteristics of RS scene,features from convolution layer and FC layer are optimized by single-type feature extraction,multi-type feature fusion and so on.Thus,key information of RS scenes are extracted to obtain discriminative representation based on transfer features.This research achieves more accurate high-resolution RS scene classification with limited training data,which fully discover the potential application value of RS image data,and satisfy the government and public needs.