Research On Deep Learning Extraction Method Of Building Structure Type Based On Multi-source Information

Buildings are the main disaster-bearing bodies in earthquake disasters.It is very significant to quickly obtain the structural type information of buildings and evaluate the seismic capacity of buildings to reduce economic losses caused by earthquakes and reduce casualties caused by earthquakes.The rapid development of deep learning technology provides a new solution to this problem.Multi-source spatial information can make up for the shortcomings of a single remote sensing image,supplement information,broaden the coverage of information contained in input data,improve the accuracy of prediction results,and the robustness of prediction models.Taking this as the research goal,the paper proposes a U-Net+DBM deep learning neural network that integrates multi-source and multi-modal information such as high-resolution remote sensing images,POI,building age,height,etc.to extract buildings and their structure type.The network model improves the accuracy and automation of extraction of buildings and their structural types.The main research contents are as follows:1.The selection of multimodal deep learning network models.The structural characteristics and application advantages of the currently commonly used deep learning image semantic segmentation network models are analyzed in detail.According to the two modal data of image and text used in the study,the U-Net neural network model is selected as the basic network model in this paper.2.The characteristics of three multi-modal data fusion methods of early-fusion,intermediate-fusion and late-fusion are analyzed.For the problem of small samples of field survey data,the network architecture of Deep Boltzmann Machine（DBM）is added to the U-Net neural network.In the network,a U-Net+DBM neural network model is constructed by late-fusion,which is used for multi-modal data fusion of images and texts to extract building structure type information.3.Based on the Beijing-Ⅱ satellite imagery and on-site survey data in Haidian District,Beijing,the multimodal sample data of four structural types of buildings including brick-wood structure,brick-concrete structure,steel-concrete structure and steel structure are established.which contains remote sensing images and building height,POI,construction age.The paper used sample enhancement,including rotation,contrast enhancement,sharpening enhancement,and color enhancement.4.Based on the constructed U-Net+DBM neural network model and sample data set,an experimental study on the extraction of buildings and their structure types from multi-source data was carried out.Considering the completeness of each attribute information in practical applications,the paper designs 8 experiments of single attribute,two combined attributes,and three combined attributes of remote sensing images and three attributes of height,POI,and construction age.Analyze and compare the accuracy,recall,F-value,IoU,MIoU of different structural types extracted by each deep learning model.The experimental results show that the fusion of multi-source spatial information including POI,age,height and remote sensing images,the neural network model can effectively improve the extraction accuracy of building structure types.The precision and recall rate of each structure category are increased by about 20%,and the IoU rate is increased by about 25%.The overall accuracy rate has increased from 66% when using only images,and increased to 85% after adding POI,age and height,and multimodal fusion extraction increased by about 20%,MIoU increased from 50% when using only images,and increased to 75% by adding POI,age and height,an increase of about 25%.In addition,the research shows that the influence of attribute information on the extraction accuracy of building structure types is: height > age > POI,and the influence of height information on steel-concrete structure and steel structure buildings is greater than that on brick-concrete structure,steel-concrete structure.In summary,the neural network model of U-Net+DBM integrating multi-source and multi-modality constructed in this paper can significantly improve the accuracy of building structure type information extraction,and has a good application prospect in building earthquake disaster risk census.