Research On Three-Dimensional Image Processing Technology Applied To Telepresence

In some dangerous areas,such as heavy nuclear radiation area,human can not work in the field,the introduction of teleoperation technology is a better solution to this problem.In operation control,the remote operator wears VR glasses to remotely control the equipment in the dangerous area for remote work.For some delicate tasks,such as grasping some complex objects,the operator needs to clearly see the 3D physical structure of the object,and also needs to know the pose of the object for the more accurate operation.But sometimes the object is blocked,it is difficult to judge the pose of the object only by the eyes.In order to solve the above problems,it is necessary to solve the problems of 3D image remote transmission,3D point cloud telepresence,object pose estimation and so on.Therefore,this paper designs three-dimensional image preprocessing algorithm,three-dimensional image remote transmission framework,three-dimensional3 D point cloud telepresence framework,object pose estimation algorithm,which are used in remote presentation.First,3D point cloud data is acquired by stereo camera at the far end.Due to the error and distortion of the camera,3D point cloud data needs to be preprocessed.In this paper,3D point cloud filter and 3D point cloud segmentation algorithm are designed to preprocess 3D point cloud data.In order to speed up the post-processing operation,the point cloud is voxeled.Experiments show that 3D point cloud filter and 3D point cloud segmentation algorithm can effectively remove outliers,make 3D point cloud smoother,and improve the performance of later processing algorithm.Then,in order to let the teleoperator work remotely in real time,this paper designs a 3D point cloud compression algorithm,which first compresses the point cloud,then transmits it on the network,and then decompresses it at the operation end.In this paper,two compression frameworks are designed,one is based on global clustering,the other is based on octree local 3D point cloud compression framework.In the first method,first,in order to further remove the redundancy of the 3D point cloud data,the temporal correlation between point clouds is acquired through employing the motion estimation.The SHOT(signatures of histograms orientation)descriptor is applied to match points between two frames for calculating the motion vector.In order to reduce calculation time,before matching operation,each frame is voxelized through using the octree structure.Then,the retained information of the Predicted frame(P frame)and the motion vectors are transmitted into the remote end,and the information of the Intra frame(I frame),the retained information of the P frame and the motion vectors are utilized to predict the new frame.Experimental results have shown the better performance of the proposed method due to the much less calculation time and the lower compression ratio.The key of the second method is to reduce the temporal redundancy through finding the temporal correlation between two frame point cloud data.The temporal correlation is acquired by the ICP algorithm.The first,the macroblock will be generated by voxelizing the point cloud data.Then,the two frame point cloud data will be aligned,and the corresponding macroblock between P frame and the I frame will be preliminarily judged whether they are consistent according to the point number of the macroblock and the color variance.The last,the ICP algorithm will be exploited to further match the two macroblocks,when the result of the ICP algorithm is lower than the Threshold,the macroblock in the P frame will be predicted by the corresponding macroblock in the I frame.In the proposed method,the size of the macroblock is variable.The experimental result demonstrates that the time delay will be reduced and the compression ratio will be increased in the proposed method,the visual result of our method also will more smooth and the distortion rate is the lower.Then,the pose of the object is estimated to facilitate the task operation.In this paper,we propose a network framework named Pose Point RCNN,to estimate the 6D pose of an object.The coordinate of the raw 3D point cloud and the RGB color information are treated as input to the network.The pose estimation is composed of two stages.In the first stage,the object's3D proposals and its pose are generated,which provides a roughly pose estimation for refining computation in the second stage.Besides refining,the attention mechanism and the mask branch are employed in order to achieve higher recall and accuracy.We randomly placed objects in different scenes to augment TUW Object Instance Recognition Dataset,then an innovative method is designed to generate the sample labels for training proposed network.The experiment results show that the object's pose can be estimated correctly with the presented method.By adding the attention mechanism and the mask branch,the proposed network achieves the recall of 53.24 compared with the recall of 16.72 without the attention mechanism and the mask branch.Finally,at the operation end,the unzipped point cloud and object pose are rendered by unity engine,and the operator can watch the remote 3D scene through VR glasses.Experiments show that the operator can clearly see the remote 3D scene and successfully operate the remote control equipment.This paper studies and designs 3D point cloud data preprocessing algorithm,two real-time3 D point cloud compression algorithms,object pose estimation algorithm based on 3D point cloud and point cloud in unity rendering framework,which provides technical support for realtime telepresence technology.