Research On Remote Operation Of Mobilemanipulator Based On 3D Reconstruction

Robot teleoperation technology is an important branch of intelligent robot field,and it is also the hotspot and focus of current research,and has wide application prospect.The introduction of 3D reconstruction to the robot teleoperation system,the operator can provide real-time scene of the three-dimensional information,and the operator easy to has a better understanding on the site environment information and improve the efficiency of remote operation.UAgent-RRS crawler-type mobile arm,is an integrated system of four cameras,four infrared PSD,photoelectric encoder and electronic compass and other sensors.When the uAgent-RRS crawler-type mobile robot is operated remotely,the operator guides the operation of the mobile arm by observing the four-way video information.However,the video information does not have three-dimensional information and can not effectively assist the operator in teleoperation.In view of this problem,the main contents of this paper are as follows:(1)A simple and fast 3D reconstruction algorithm based on Kinect is designed and implemented.Firstly,each frame of the depth of information and color images is obtained based on the Kinect sensor.Secondly,Based on an improved joint bilateral filtering algorithm,the depth information is filtered to deal with the noise present in the depth information image and compared with the effects of median filtering,Gaussian filtering,bilateral filtering and combined bilateral filtering.Thirdly,the environmental information point cloud is obtained based on the depth of the image processing,building a matrix with the same size as the depth map,used to store the three-dimensional data of the point cloud.Forthly,Based on the orderly point cloud for a simple fast triangulation,the use of the characteristics of ordered point cloud,scanning all the points in the cloud,that point in the cloud point of each point and the relationship between all the points,so that point cloud In each of the points with its previous two points,constructed into a triangular mesh.Lastly,Based on the least squares method to set the vertex normal vector for each point in the point cloud,construct a three-dimensional effect.The algorithm is based on the general three-dimensional reconstruction of the process framework,can be simple and fast real-time reconstruction of the scene of the three-dimensional information.(2)Design and implementation of the mobile arm remote operation of theexperimental platform.According to the sensor data acquisition,the body,arm,PTZ control,image transmission of the three modules of different functions,based on the wireless communication protocol used in this paper,respectively,the host computer software MFC design.(3)Based on the the mobile arm remote operation of the experimental platform and Kinect-based three-dimensional reconstruction algorithm,a remote control system based on virtual reality is designed and built.Using this system,the experiment of capturing the target object by teleoperation mobile manipulator is realized.In this paper,the experimental results show that the method used in this paper can better assist the operator in the remote operation of the mobile manipulator.