Semantic SLAM Based Perception In Escort Robot Scene

Home-based care is an important way to solve the problem of providing for the aged in China,the escort robot will play an important role in home-based care.This paper studies the key technologies of scene perception of home care robot,including scene recognition,robot positioning and human-computer interaction.First of all,realize semantic segmentation based on machine vision scene to identify the category and location of objects in the scene;then,improve the robustness and accuracy of location of the escort robot in the dynamic family scene by combining the scene perception information;finally,in terms of the robot's escort function,combine the field semantic information to study the human-computer interaction function,so that some high semantic human-computer interaction tasks become to be possible,such as picking,carrying and other operations,at the same time,study the detection methods of the elderly's abnormal state,such as falls,abnormal living habits monitoring,etc.,to protect the life safety of the elderly.The main research work of this paper is as follows:Firstly,a scene perception model based on depth convolution neural network is proposed,which is a depth prediction network based on RGB image and a semantic segmentation network based on RGBD image.Through depth prediction and semantic segmentation,the robot can get the categories and relative positions of objects in the scene,which endows the robot with the ability of scene perception in the semantic level.Secondly,five robot positioning optimization methods are proposed,which are semantic constraint,depth constraint,HSV feature point matching,dynamic suppression and seman-tic loopback detection.First,semantic segmentation and depth prediction algorithm are used to improve the dimension of robot's perception of scene information.At the front of vision SLAM system,semantic constraint and depth constraint are used to suppress mismatched fea-ture points.Second,semantic information is used to assist loop back detection to improve the stability of system algorithm.Final,combining image semantic segmentation technology and dynamic detection technology,dynamic detection technology is used to dynamically sup-pressed.Through the above methods,the positioning robustness and accuracy of the home care robot are improved.Thirdly,combining the point cloud information and semantic information,construct the semantic map of the scene,and propose a semantic human-computer interaction method in the aspect of "accompany" According to the instructions of the elderly,simulate and realize the independent pick-up or handling of items,and assist the elderly to complete the basic tasks of family life self-care.In the aspect of"protect",there are two situations:one is to identify the elderly and extract them through dynamic tracking.Its current skeletal information,combined with semantic information and skeletal feature change information,judges the possibility of the elderly's falling risk;second,by recording the elderly's daily life habits,judges the possibility of the elderly's abnormal from the time and space.Finally,combined with the research content and research methods proposed above,build an experimental platform for home care robot,and build the corresponding hardware platform and software platform,and carry out experiments on scene perception,robot positioning and map building.The experiments show that the content and methods studied in this paper are practical and feasible.The innovation of this paper is as follows:Firstly,a RGBD semantic scene perception method based on monocular camera is pro-posed.Using the predicted depth data D and image RGB to form a RGBD four channel image,the accuracy of the semantic segmentation model is improved,and the escort robot can sense the scene at the semantic level.Secondly,based on semantic SLAM,an optimization method of robot location in dynamic scene is proposed.Aiming at the dynamic changes of people,objects,illumination and the movement of the robot itself in the family environment,the methods of semantic constraint,depth constraint,dynamic constraint,HSV feature matching and semantic word bag are used to improve the positioning robustness and accuracy of the escort robot.Finally,a semantic human-computer interaction method is proposed.Combined with the research of semantic segmentation and robot positioning technology,the semantic map of the scene is constructed and the experiment of item category retrieval,simulated grabbing and carrying in the family scene is realized.