Human-Robot Interaction Based On 3D-Mapping And Virtual Reality

Human-Robot Interaction(HRI)usually uses buttons or others as input devices of control commands,and cameras as information output devices.This way has a series of disadvantages,for example,low efficiency,time-delay,lack of telepresense,etc.With the development of Virtual Reality(VR)and computers,it is possible for the operator to provide efficient HRI system with a strong sense of telepresence and small time-delay.Aiming at improving the level of HRI,in this thesis,a HRI system based on 3D mapping and visual reality is proposed and implemented.Firstly,a HRI system is proposed based on the VR.This paper introduces and analyses the working mechanism of the proposed system form the robot system and the VR system.Then,the hardware devices and the software,involved in the HRI system,are introduced,and analyses all modules' functions and informations flow.Finally,the coordinates used in the proposed system and their relationships are defined for the unified and convenient representation.which offers the theoretical foundation for the realization of the HRI system based on the VR.Secondly,the LOAM algorithm is used to construct the 3D point map and estimate the robot's pose in the indoor and outdoor scenes,and this thesis improves the performance of the LOAM algorithm by two methods,which are the fusion of the LiDAR data and inertial calculation based on the Extended Kalman Filter(EKF)to further improve the estimation accuracy of the robot's pose,and coupling LOAM algorithm and 3D-NDT algorithm to improve the registration accuracy.Ulteriorly,the robot's 3D environment and pose are visualized in the VR system in real-time.Because of the large amount of data of the 3D point map,the incremental 3D-NDT is used to decrease the data of the 3D point map.Then the map and the robot's pose are transmitted and visualized in the VR system.The system can visualize the robot's 3D environment and pose in the VR system in real-time,and provide a strong telepresence for the operator.Finally,three HRI systems are proposed based on the visualization of the robot's 3D environment and pose in the VR system and the mission to explore the unknow environment can be realized for the human and robot together.Respectively,1)the robot can be controled by the speed commands generated by the operator;2)the robot can be controled by the path commands generated by the operator;3)the robot can be controled by the target commands generated by the operator.The experimental results show that the oprator can interacts efficiently with the robot by distributing speed commands and specifying the path and the target in the proposed HRI system.