Research On The Unstructured Environment Reconstruction Of Vitrual Reality Assisted Telerobotic

With the rapid development of robot technology, the presence-basedteleoperation robot is playing an increasingly important role in unknown or dangerousenvironments. In order to realize effective control of the remote robot, the operatormust fully understand the working environment of robots and get the accurate relativeposition relationship between the robot and objects or the environment.Three-dimensional reconstruction of the operating environment is an effective methodto understand the operating environment, the accuracy of reconstruction models is thekey to the success of teleoperation tasks. So, how to build accurate and real virtualscene depending on the site environmental information is the primary problem. In thetraditional virtual reality assisted robotic teleoperation, the virtual environment isoften pre-modeling, which is unable to adapt to unpredictable or changingenvironment. The recognition and reconstruction of the environment by multi-cameracan realize dynamic environment modeling, but it is easily lead to false matchingpoints on image matching. Meanwhile, by the limitations of image size, only partialscene can be reconstructed and the reconstruction of large-scale scene can’t beachieved.Combined with National Natural Science Foundation of China “Research onshared control strategies for robot system based on kinesthetic and force guidingmethod” and key scientific research projects of Jilin province “Kinesthetic guidingshared control for robot system”, the paper completes the3D reconstruction of theoperating environment for telerobotics by using the unstructured environmentreconstruction method based on the three-dimensional point cloud data. By combiningthe visual information of RGB camera and infrared camera, the three-dimensionalpoint cloud data of operating environment, which contains the color information, canbe fast got. To get the complete scene data, an improved ICP algorithm is used toregistration the point cloud data from different views. In order to separate the jobobject and operating environment, the paper introduces the region growing segmentation algorithm that is based on the geometric information and colorinformation. Based on the Delaunay triangulation algorithm, the3D virtualenvironment of robot operating environment is been reconstructed from thethree-dimensional point cloud data.The main tasks completed by this paper are as follows:1) The paper uses D-H method to solve the position and orientation of robotends, and calibrates the hand-eye system to get the hand-eye relationship. Then, basedon the relationship between each coordinate system in robot system, a description ofthe scene in robot base coordinate system is established.2) The internal and external camera parameters of cameras are obtained by thecalibration of RGB camera and infrared camera. Then by combining the visualinformation from RGB camera and infrared camera, the point cloud data that containscolor information can be obtained.3) In order to remove noise, outliers and redundant points, the paper performssome pre-processing methods, such as smoothing, reduction, surface normalsestimation and so on. Thus it can provide the foundation for subsequent point cloudprocessing operation.4) To get the complete scene data, an improved ICP algorithm is introduced toregistration the point cloud data from different views. Meanwhile, for the data afterregistration, re-sampling process can obtain a uniform smooth surface data.5) In order to separate the job object and operating environment, the regiongrowing segmentation algorithm performs well which is based on the geometricinformation and color information.6) Based on the Delaunay triangulation algorithm,3D models of differentsegmentations are reconstructed independently and then all reconstruction models areplaced in the same scene. Finally3D virtual environment of robot operatingenvironment is been reconstructed.