Research On Remotely Sensed Image Scene Deep Learning And Application

The traditional land use/cover information extracted from remote sensing data includes the pixel-based information of land cover type,but lacks of the structural information in local neighborhood information with a larger scale.In terms of the urban region with complicated spatial-layout,the pixel-based land use/cover information contains relatively few categories without sufficiant spatial information to represent the complicated spatial-layout of land surface.Therefore,there is a problem of information asymmetry existing between the land use/cover information extracted from remote sensing data and the abundant spatial information of urban surface,which seriously restricts applications of remote sensing technology in urban related studies.As a result,how to extract abundant spatial information of urban region from remote sensing data becomes the key factor to improve applications of remote sensing technology in the study of urban area.High spatial resolution remote sensing technology can collect imagery of land surface including detailed land cover and abundant local structural information,so it provides possibility to extract spatial information of complicated urban surface using remote sensing data.Hence,this work focuses on the requirement of extracting spatial information of complicated urban surface in the National Natural Science Foundation of China "time series remote sensing image intelligent processing and temporal-spatial analysis for the geographic process" and Jiangsu Provincial Natural Science Foundation"multiscale remote sensing information cooperative processing and urban human settlement evaluation".It attempts to explore the new ideas and approaches to extract spatial information of urban region based on high-spatial-resolution(HSR)remotely sensed imagery.With respect to the representation and processing unit of complicated spatial information for the HSR imagery,scene was selected as the elementary unit to represent and process spatial information.This study focuses on the key techniques and approaches of feature extraction and representation based on deep learning method,and their application on urban structure type recognition.It was carried out along a main line of "deep learning-feature coding-feature fusion",and proposed more suitable feature representation methods for remotely sensed scene images.Then,the feature extraction and representation methods were applied to urban structure type recognition based on scene unit,and it achieved accurate recognition for the spatial information of urban structure types.In this study,the performances of feature learning and representation methods for remotely sensed imagery scenes were evaluated via two typical remotely sensed scene datasets UCM and WHU_RS.Finally,two HSR imageries collected by IKONOS-2 and SPOT 7 sensors were used to conduct the application study of urban structure recognition in the study area of Nanjing.Specifically,the main research contents and conclusions include the following four parts.1)This dissertation has explored the basic flows and key technologies of feature learning and representation based on deep learning for remotely sensed scene.On the basis of two typical deep learning frameworks(i.e.,sparse autoencoder and deep convolutional neural networks),it proposed or improved corresponding feature coding methods for the convolutional features extracted by different deep learning methods.Scene feature learning and representation is the first problem to be solved for scene-based spatial information extraction,which aims to represent the scene image by a feature vector.Deep learning,one of the most potential methods in the computer vision community recently,can complete scene feature representation by capturing local information of scene images based on feature learning process.However,the convolutional features extracted by deep learning methods need to be linearly represented by feature coding methods to represent the scene images.For the dense convolutional features extracted by sparse autoencoder,a global feature coding(GFC)method was proposed and it has a powerful ability of feature representation with the dimensionality decrease of convolutional features.For the convolutional features extracted by deep convolutional neural networks,a multiscale improved Fisher kernel(MIFK)coding method was developed and the feature representation ability of the method was improved through taking the variousness of image scale into consideration.The deep learning and feature coding methods were evaluated by the standard remotely sensed scene datasets.The experimental results indicated that the proposed methods was effective,and the other important discovery was that the deep convolutional neural networks has a greater advantage than the sparse autoencoder in the feature learning and representation for remotely sensed scene.2)Deep convolutional neural networks can extract multi-layer features of the scene image,and the feature representation ability of image scene could be efficiently improved by fusing multi-layer features of scene image.Different layer features contain different information of the image scene.Moreover,they have obvious differences and a certain complementarily.Nevertheless,the coded features of different convolutional layer features and the features extracted by fully-connected layers have very high dimensionality.Aiming at this problem,a feature fusion strategy was proposed based on supervised feature subspace learning method to fuse multiple high dimensional feature data.The experiments concluded that the proposed method of multi-layer feature fusion greatly improved the feature representation ability,and obviously improved the scene classification accuracy.3)Due to the obvious difference and complementarily of model architecture and parameter existing in the various deep convolutional neural networks,it could further improve the feature representation ability of the image scene via fusing features from different models,which is more effective for the task with limited labeled samples.The feature fusion method developed based on supervised feature subspace learning can effectively fuse multi-layer features from a single model to complete the feature representation.However,it also reduces the difference existing among the fused features of different models.Hence,the feature fusion method can be improved by simultaneously utilizing the supervised and unsupervised feature subspace learning methods.The improved fusion method can preserve the features difference of various models,and becomes the foundation to fuse features from multiple models.Experimental results indicated that the improved fusion method has effectively balanced the relationship between the feature difference of multiple models and their discrimination.As a result,it obviously improved representation ability of fused features especially for the case with limited labeled samples.4)On the basis of semantic definition for urban structure types,scene was used as the processing unit,and the feature representation methods based on deep learning for remotely sensed scene images were applied to urban structure types recognition,which makes the remote sensing information more satisfying the requirement of urban related studies.Through the applications of urban structure type recognition in the study area of Nanjing using two different kinds of HSR remote sensing imagery,it indicated that the proposed feature representation framework and methods for scene images was feasible.It obtained classification accuracy of over 90%for urban structure type recognition in in the study area using the proposed methods of fusing multi-layer features and multi-model features.In addition,the results shows that the method of fusing multi-model features have competitive advantage for image scene feature representation with limited samples.Moreover,this study is of real application and reference values in extracting neighborhood scale spatial information from HSR imagery and provide a new idea to improve the application of HSR imagery in the study of urban area.