Research On Construction Of Indoor Semantic Map Based On Deep Learning

With the development of social economy and the improvement of people’s living standard,the intelligent level of robot is required to be higher.The core factor that affects the intelligent level of robot is the autonomous perception and understanding of the surrounding environment information.The robot can reconstruct the geometric model of unknown environment through SLAM.However,the model lacks more detailed explanation of the object attributes and functions in the environment,which makes the robot unable to fully understand the scene information of unknown environment,so it is difficult to complete more advanced human-computer interaction tasks.In recent years,the improvement and development of deep learning technology has provided a new opportunity for improving the robot’s environmental perception ability,which has greatly promoted the improvement of the robot’s intelligent level,and thus has provided the basis and guarantee for the robot to complete more advanced human-computer interaction tasks.However,to realize the real-time construction of high-precision semantic map in complex indoor environment with a wide variety of objects and serious occlusion becomes the main challenge of the current semantic map construction system based on deep learning.Thus,this paper studies the construction of indoor semantic map based on deep learning,the works of this paper is as follows:(1)The system comprehensively reviews the research status of the construction of indoor semantic map based on deep learning from three aspects of visual SLAM,image semantic segmentation and semantic map construction,and points out the shortcomings of the existing systems from the aspects of real-time,robustness and semantic annotation accuracy.(2)Aiming at the problem that it is difficult to achieve accurate semantic segmentation of indoor complex scene images with various objects and serious occlusion only by image visual color information,this paper studies and proposes a indoor rgb-d image semantic segmentation network based on multi-scale feature joint,which can simultaneously extract the visual color features and depth geometric features of rgb-d images and effectively fuse them layer by layer,so as to achieve accurate semantic segmentation of complex indoor scene images.In this paper,SUN RGB-D and NYUD v2 indoor standard datasets are used to qualitatively and quantitatively evaluate the performance of the image semantic segmentation network proposed in this paper.The experimental results show that the indoor rgb-d image semantic segmentation network based on multi-scale feature joint proposed in this paper can obtain accurate segmentation results for a variety of indoor complex scene images.(3)Aiming at the problem of insufficient semantic annotation precision and efficiency of the current indoor semantic map construction system,this paper studies and proposes a semantic map construction system of indoor scene based on deep learning.The indoor rgb-d image semantic segmentation network based on multi-scale feature joint proposed in this paper is combined with the visual SLAM based on rgb-d camera to realize the real-time construction of high-precision semantic map of indoor environment.The indoor semantic map construction experiments was carried out in the laboratory.The experimental results show that the semantic map construction system proposed in this paper can process the image sequence of the surrounding environment collected by the sensors carried by the mobile robot online and real-time,and obtain the accurate semantic segmentation results of the single frame indoor scene image and the currently constructed local spatial semantic map at the same time.Ultimately,a globally consistent and high-precision 3D semantic map of real indoor scene is obtained.