Research On Visual Perception And Augmented Reality Technology Of Rescue Robot For Human-Robot Interaction

When using remote control robot to perform rescue tasks,the general practice is to use the robot camera to directly return the image information for the operator to observe the environment.In this case,the operator needs to concentrate on finding the victim,and remember the path the robot has traveled and the current position of the robot,consuming a lot of operator energy and causing fatigue easily.This paper considers the use of augmented reality to provide more information on the image,enhance the operation experience,and reduce the operator requirements.Firstly,an augmented reality system framework for rescue robots is proposed,where an augmented reality display scheme classified as "video perspective" is adopted.The video information returned by rescue robot is regarded as the real world,and the work of augmented reality is completed on the digitized video stream.Then the hardware and software system of the system are described,and the four elements of the augmented reality system are introduced,which is the basis of the follow-up work.Secondly,we develop the software framework of augmented reality system with different nodes.Robot Operating System(ROS)is used as the foundation for building this system.Then,the basic technology of realizing augmented reality in unlabeled environment is introduced.ORB-SLAM2,which is used to realize pose tracking for the augmented reality system,and the method of realizing real environment modeling and display annotation using SLAM nodes,are introduced.Next,we introduce two methods of target detection and localization using deep learning method,and finally choose OSVOS target segmentation network.The information of each node is integrated and processed in the AR master node to realize the ultimate augmented reality display and assist operators to mark targets in the world.Finally,we design a multi-modal map construction framework to build the point cloud map,gridmap and costmap.Our framework relies only on RGBD sensors,enabling the system to have good portability.This paper introduces the construction of dense point cloud map.The dense point cloud map can restore the real environment well,but the obstacles still need to be judged by operators.On the basis of point cloud map,the ground point cloud and obstacle point cloud are segmented using the point features related to normal vector in PCL library,and the obstacle grid map is established.Finally,the cost map is constructed using pseudo laser data generated by RGBD camera,and the robot path is planned for operator's reference.Experiments show that the multi-modal map can help operators understand environmental information more conveniently.