Creating An Effective Geometry-aware And Image Understanding Network For Automatic Building Extraction From High Resolution Image And LiDAR Data

The automatic extraction of building information from remote sensing data is of considerable significance and value in cartography,3D reconstruction,and urban change analysis.In recent years,the rapid development of earth observation technology has dramatically improved the quality and update speed of remote sensing data,making multi-source remote sensing data widely available,such as high-resolution remote sensing image and 3D Li DAR data,which provide opportunities for more accurate extraction of building information.However,the multi-source remote sensing data with redundant information and intricate details also brings a series of problems and challenges to the interpretation accuracy and reliability of the automatic building extraction method in complex urban scenes.In recent years,the full convolution neural network based on deep learning has its strong ability to feature representation and has achieved excellent end-to-end pixellevel classification results in building extraction.At present,it is an effective way to improve the accuracy and reliability of building extraction results by using a full convolution neural network to learn the building features in multi-source remote sensing data.The conventional processing method is to extract the digital surface model based on the 3D Li DAR data as the network auxiliary or additional input features.Such a processing strategy not only lacks the full mining of spatial geometry information in3 D Li DAR data but also dramatically increases the amount of calculation and parameters of the network.Besides,the output classification results lack detailed information,which is the main problem faced by the structure of the full convolution neural network.When extracting a small building object from a complex urban scene,the problem of the fuzzy boundary will be more prominent.Because of the above problems,this paper proposes an efficient geometric-aware and image understanding semantic segmentation network.The main research ideas and innovations are as follows:(1)According to the difference of remote sensing data sources used in building automatic extraction task,this paper fully describes the development process,and related theories of traditional building extraction methods,as well as building extraction methods based on deep learning and sums up limitations and challenges in these methods,which provides necessary theoretical support and development for the network structure and related modules proposed in this paper.(2)The encoder part of the network model is based on the deep residual network added with an efficient building local height-aware structure.This structure can flexibly and efficiently integrate geometric information into the network model without increasing the amount of calculation and parameters.The network model with this structure successfully overcomes the problems of low learning efficiency and a large amount of computing resources consuming.Also,in this paper,an efficient building 3D geometric-aware structure is integrated into the encoder part of the network.This structure is mainly based on the mimic 3D convolution operation.It not only computing friendly but also strengthens the mining of geometric information in the 3D Li DAR data by the network model.(3)In this paper,an efficient multi-scale aware module is added between the encoder and decoder part of the network,which effectively expands the network perception field and improves the learning ability of the network to the multi-scale features of the building objects.At the decoder part of the network,this paper combines the multi-level features learned from the network to recover the size of the input image gradually,which effectively solves the fuzzy problem of building classification results in the full convolution semantic segmentation network.Based on the open-source datasets of different complex urban scenes,this paper takes a lot of comparative experiments between the proposed network model and the state-of-art automatic building extraction methods.Besides,in order to verify the effect of proposed efficient building local height-aware structure,the efficient building 3D geometry-aware structure and efficient building multi-scale aware module,this paper carried out many ablation studies with some similar structures/modules on test datasets.Compared with other state-of-art network models,the results indicate that the proposed end-to-end semantic segmentation network can effectively learn the geometric features and image features of building objects in multi-source remote sensing data and achieve the new state-of-art building extraction results.