Semantic Segmentation Of High-Spatial-Resolution Remotely Sensed Imagery

This study focused on semantic segmentation of high-spatial-resolution remotely sensed(HR)imagery.Semantic segmentation aims at obtaining multiple levels of spatial semantics in HR images,which is closely related to the objectives and requirements of the National Science and Technology Major Project "snow and ice monitoring and its evaluation based on high-resolution remote sensing data in central Tianshan Mountains in Xinjiang"(Grant NO.95-Y40B02-9001-13/15-04)and the National High Technology and Development Program "New segmentation technology for high spatial resolution satellite imagery"(Grant NO.2008AA12Z106).HR images provide a great amount of details about the earth surface.The detailed information of earth surface can be transferred into spatial semantics at multiple levels by inference.Extracting the abundant semantics and detailed land cover information from HR images is a key step to applying HR images.However,the increased spatial resolution brings new challenges to information extraction from HR images,and at the same time the semantic gap between low-level image features and high-level semantics greatly limits the ability of obtaining semantics from HR images.The PMS data of GF-1 satellite was utilized in this study.The study area was located in the plain area of Manasi River Basin,involving the Shihezi City and its surrounding area.To obtain the rich spatial semantics in HR images,the semantic hierarchy in HR images was first defined,including the levels of image features,object semantics,and scene semantics.The semantic databases were then built at each of the three levels,based on which,a set of methods were developed to perform multiscale HR image segmentation,object semantic segmentation,and scene semantic mining.Specifically,the study contents and results include the following five parts.First,the spatial semantic hierarchy in HR images was defined,laying the theoretical foundation for semantic information extraction from HR images.A 10-level semantic hierarchy was defined based on the involved knowledge in the interpretation.Only the first six levels were interpreted in this study and they were defined as levels of image features,object semantics,and scene semantics.Images features included the spectra,texture,shape,etc.,which can be directly calculated based on images.The object semantics included the type,boundary,and location of objects in HR images.The scene semantics indicated the scene type information,which was expressed by the objects and their spatial distribution within HR images.Second,the semantic databases at the levels of image features,object semantics,and scene semantics were built,which can provide data and prior information for extracting semantics from HR images.The image feature database was actually an algorithm set,where a set of algorithms to calculate features based on pixels,windows,lines,and regions was integrated,serving as the basis for obtaining.semantics.The object semantic database had 18 GF-1 images in the study area,representing the landscapes of urban residential area,school,factory,and rural area.The types and boundaries of all the objects in each image were manually labeled,which can be used for learning parameters and evaluating results of semantic segmentation methods.The scene semantic database had 150 images with scene types of urban residential area,rural residential area,and factory area.There were 50 images for each scene type.This database was used to train and evaluate scene semantic mining models.Third,a set of multiscale image segmentation methods was developed for HR images,aiming at providing multiscale segmented regions and segmentation methodology for successive object semantic segmentation.The region merging method was adopted,in which several key steps were specially studied,including the graph model,merging criterion,merging strategy,scale parameter,multiscale segments representation,and segmentation evaluation for both single-scale and multiscale segmentation results.In terms of the graph model,a linear nearest graph model was proposed to accelerate the segmentation procedure.The adaptively increased edge strength was integrated in merging criterion to make the segments closely related to real objects.A hybrid region merging strategy was developed by combing the strengths of both global-and local-oriented region merging strategies.In the region merging process,the adaptive increased scale parameters were set to produce nested multiscale segments and to make the multiscale segmentations cover meaningful segmentation scales.A segment tree was used to represent multiscale segments,which can produce multiple segmentations very efficiently.The precision and recall measures were proposed to reveal both the geometric and arithmetic errors of single-scale segmentation result.A multiscale object accuracy measure was proposed to reveal the quality of multiple segmentation results.Forth,three kinds of object semantic segmentation strategies were proposed by combining prior information with image features.The proposed strategies included the combination of prior information and multiscale segmentations,prior-imbedded multiscale semantic segmentation,and semantic segmentation based on the conditional random field(CRF).They can get the spatial semantics of objects by determining the types and boundaries of objects concurrently and can avoid the problem of scale selection in the object-based image analysis(OBIA).The first strategy combined multiscale segmentations represented by a segment tree with prior information by the max-sum message propagation algorithm,which can achieve global optimization and determine object boundaries and types by utilizing multiple segmentations automatically.The multiscale semantic segmentation strategy integrated prior information into the multiscale image segmentation framework.Then,both the image structure information and prior information were considered in the spatial optimization process,which made the segmentation scale directly related to object semantics.Two CRF-based semantic segmentation strategies were developed:single-image and multiple-images oriented,both of which can model the spatial interactions of prior information and image structural information by the unary and pairwise potentials.CRF can simultaneously generate object types and boundaries by inference with global optimization,which can also avoid the problem of scale selection.Specially,the CRF-based semantic segmentation strategy for multiple images can interpret unknown images by the learned model,which can improve the ability of processing large numbers of remotely sensed images.In the end,a Bayesian network model was developed for scene semantic mining by utilizing object semantics.The network structure was determined by the specific images in the scene semantic database.The object features of concept occurrence probability,geometric features of single objects,and spatial relations between multiple objects were integrated in the model.The nodes at different levels in the network represented the links between object features.The maximum a posterior of the root node of the network was calculated to indicate the scene type.The scene semantic database was used to learn the parameters of the model.The test results proved the effectiveness of the object semantics and the proposed Bayesian network model for scene semantic mining.Furthermore,the results showed the necessity of combining concept occurrence probability and spatial features of objects for understanding scene semantics.As a whole,this study proposed a semantic segmentation framework by first determining semantic hierarchy,and then performing object semantic segmentation,and finally mining scene semantics.The proposed framework can get multi-level spatial semantics in HR images with the support of the proposed semantic databases,which will help to result in abundant applications of HR images.The proposed object semantic segmentation strategies went beyond the bottom-up image segmentation strategy and avoided the problem of segmentation scale selection by combining the prior information with image structural information.They generated object types and boundaries simultaneously,which will have the potential to be used for information extraction from HR images.The proposed Bayesian network model can be utilized for scene semantic mining based on object semantics.The network was able to integrate numbers of object features and geographic knowledge for scene understanding,which will help to push forward the abudent applications of HR images.