RGBD Semantic Segmentation Method Based On Deep Learning In Complex Indoor Scene

In complex indoor environment,mobile robot environment perception is susceptible to interference such as uneven indoor light and irregular occlusion of various targets,which affects robot autonomous operation.Therefore,this paper in order to improve the indoor environment robot semantic segmentation ability,for the purpose of key research RGBD semantic segmentation method based on the deep learning,through the environment of RGB texture information and environmental information of the structure under effective fusion depth data,implement semantic block and light not equal precise semantic segmentation under complex environment.The specific work of this paper includes:(1)Proposed a RGBD semantic segmentation method based on structural similarity fusion.This method first designs the local structure similarity representation matrix of RGB and depth map,then weights them based on the structure similarity matrix,then designs the network based on the semantic segmentation network of codec structure,and discusses the way of the fusion operator loading in the network.Finally,the ablation experiment and comparison experiment were carried out on SUN RGBD open source dataset.The results show that the improved method with the introduction of link-RGBD operator achieves significant improvement in semantic segmentation performance without increasing the number of additional parameters.Compared with the former method,the pixel segmentation accuracy is improved by 2.3%,the average segmentation accuracy is improved by 1.6%,and the average segmentation intersection ratio is improved by 3.5%.(2)An improved RGBD semantic segmentation method based on visual attention is proposed.Firstly,the visual attention module is introduced into the RGBD semantic segmentation backbone network to selectively allocate the weight of RGB features and depth features to enhance the mutual fusion of multi-channel features.Secondly,different attention embedding methods are designed and trained based on freezing training strategy.Finally,experiments are carried out on NYU-Dv2 open source dataset,and the results show that attention network can improve the performance of semantic segmentation to a certain extent.(3)The RGBD semantic segmentation application verification system for mobile robot grasping is built.Under the background of mobile robot communication node control in the indoor environment,an experimental platform of mobile robot grasping and detection based on RGBD depth camera and robot arm was constructed,and the application of data set collection and target semantic segmentation algorithm was verified.Experiments show that the RGBD semantic segmentation method designed in this paper can be carried on the mobile robot platform to achieve the semantic segmentation of typical targets,and effectively support the robot arm to complete the target grabbing and placing action.