Remote Sensing Image Retrieval Based On Multilevel Semantic Information

With the advent of the information age,the Earth observation technology has also entered a stage of rapid development.Among them,remote sensing technology has developed rapidly in the past 10 years and has been widely used in various fields,such as agriculture,ocean(vessel detection,oil spill detection),urban planning and manage-ment,water quality monitoring,ecological environment assessment,global warming,global forest,resource assessment.Alongside the rapid development of remote sensing platforms and sensors,the volume of remotely sensed imagery has also tremendously increased.Because of the large data volumes,the exploration and information mining from remote sensing archivesis becoming increasingly difficult.Obtaining the appro-priate imagery with high quility has become the key issue.Remote sensing image retrieval technology could allow the users to obtain image from massive data.Bene-fitted from this,the researchers can use the images of interest to conduct research for specific applications.Furthermore,this can improve the information mining capability of remote sensing data and ultimately improve the utilization of remote sensing images.This thesis focuses on the content-based remote sensing image retrieval technology,and introduces the remote sensing image retrieval method based on the low-level features(Bag of Words model,BoW),pixel-level semantic information and object-level semantic information.Based on deep learning,we proposed the frameworks of pixel-level informa-tion extraction for both high spatial and hyperspectral images.Moreover,we designed the feature extraction approaches for pixel-level and object-level semantic information,based on which,a remote sensing image retrieval system has been constructed.The main contributions of this thesis are as follows:(1)This paper systematically introduces the feature extraction method,similarity Imeasurement algorithm and retrieval performance evaluation index commonly used in remote sensing image retrieval based on the low-level features.The perforinane of dif ferent feature extraction methods:SIFT and Dense SIFT for Bag of words model has been analyzed and discussed.In addition,Dense SIFT and spatial segmentation strat-egy are combined into the BoW model,and remote sensing image retrieval is conducted based on this.(2)The application of the current deep learning method in the extraction of pixel-level semantic information of remote sensing image is introduced.The principles of the commonly used deep learning model and their respective characteristics and application directions are expounded.In particular,the module and its function of the convolutional neural networks are discussed and analyzed.For high-resolution images,a pixel-level semantic information extraction model based on convolutional neural network has been constructed,and the role of Batch Normalization in convolutional neural networks has also been discussed and analyzed.(3)The hyperspectral image classification model based on convolutional neural networks(CNNs)are constructed in both the spectral dimension and spatial dimension respectively.The performance of spectral dimension and spatial dimension information on hyperspectral image classification is analyzed and compared.For achieving better classification result,we try to combine spatial dimension and spectral dimension infor-mation together to construct the spectral-space CNNs.Aiming at the problem of high dimensionality in hyperspectral image information extraction,a fast density peak clus-tering band selection method based on structural similarity(SSIM-FDPC)is proposed.Compared with FDPC and EFDPC methods,SSIM-FDPC has much more robust and better performance in band selection.Then,the sensitivity of the three factors of struc-ture,contrast and brightness in the SSIM-FDPC method is discussed.Finally,the classification performance of different hyperspectral dimensionality reduction methods combined with CNNs in hyperspectral images is evaluated.Moreover,we analyzed the influence of different feature dimensions on classification accuracy and compare the stability of different dimensionality reduction methods.(4)For remote sensing image classification map,a feature extraction method based on pixel-level semantic information is proposed.Based on this,a complete remote sensing image retrieval system is constructed,which allows the user to select the region of interest,of any shape and size as the reference scene without having to be fixed size or regular shape.In addition,in order to enrich the search results and facilitate have an overview of the search area,the scenes with overlapping regions are combined and displayed together.Furthermore,to locate the search scenes in google earth,we calcuateed the geographic coordinates of the scenes according to the adopted "projection method".In order to improve the retrieval efficiency and meet the needs of retrieval on different scales,the strategy of constructing the base signature libary and signature flatten method were proposed.(5)Two object-level semantic information extraction methods are introduced,one is the traditional method based on image segmantation(direct method)and the other is based on the pixel-level semantics(indirect method).Based on this,the similarity measure based on mutual information entropy is used to compare the similarity between scenes.Taking high-resolution remote sensing image as an example,we completed the set of process from "pixel-level semantic information extraction" to "image retrieval based on object-level semantic information".Firstly,we extracted the pixel-level infor-mation based on deep learning models.Then the object-level semantic information was extracted from the classification map.In the end,we constructed the remote sensing image retrieval system based on object-level semantic information.