Research On Teleoperation Of Manipulator Based On Imitation Learning

Teleoperation technology is of great significance to realize human-computer interaction.However,the traditional teleoperation technology usually controls the manipulator through the joystick and control panel.In the operation task,the operator's experience and technology requirements are very high,and it can't imitate the human action to complete the task.For the inexperienced operator,this paper proposes two teleoperation modes based on wearable device and VR device,respectively realizing the imitation learning operation of Baxter manipulator based on probabilistic motion primitive and shadow dexterous hand based on novel teachnet deep learning network.The specific research contents are as follows:Firstly,aiming at the problem of delay in the teleoperation of wearable devices,the mapping relationship between human joints and manipulator joints is given.The selfdeveloped data glove is used to collect the posture information of the manipulator arm in real time,and the speed based control method is used to make the motion process of Baxter manipulator very smooth.Aiming at the problem of distance limitation in the remote operation of wearable devices,a wireless communication module is added to the data glove,and the communication system between the wireless communication module and the robot operating system ROS is designed.The experimental results show that the manipulator can complete multiple trajectory actions,realize the real-time motion tracking between the manipulator and human actions,and can complete the remote operation task within a certain distance range.Secondly,aiming at the problems of single data,limited distance and poor targeting of wearable devices,a teleoperation scheme based on vr virtual reality technology is proposed.The remote operation teaching of Baxter manipulator is completed by using HTC vive Pro equipment.The unity project is designed to obtain the attitude information of two handles of HTC vive Pro equipment.The communication with unity platform and the control of manipulator are completed in ROS system.Due to the joint angle control mode used in the teleoperation of wearable devices,human beings need to adjust the arm posture continuously based on experience to complete the grasping operation of the end fixture of the manipulator.Therefore,a time-based difference incremental control method is further proposed to realize the high-precision target grasping task of the manipulator.The experimental results show that the remote operation based on vr virtual reality technology has higher precision than that based on wearable devices,and can complete complex operation tasks well.Finally,in order to realize the task of piano playing jointly by manipulator and dexterous hand,different simulation learning algorithms are proposed for manipulator and dexterous hand respectively.Since vr virtual reality teleoperation needs to rely on the experimental environment and operators,a simulation learning method based on probabilistic motion primitives is proposed.By learning the trajectory data obtained from the high-quality motion samples of Baxter manipulator teleoperation,the simulation learning task is completed.Compared with the initialization work of VR equipment,such as adjusting the positioning camera position for different tasks each time,the proposed method can generalize the corresponding trajectories for different tasks.The experimental results show that not only similar task trajectories can be simulated remotely,but also similar task trajectories can be simulated for different types of ur manipulators.Furthermore,a vision based deep learning network(teachnet)shadow dexterous hand simulation learning method is proposed.The experimental results show that the error of dexterous hand simulation operation is smaller when using this framework,and the performance is better when different operators perform dexterous hand simulation tasks.At the same time,the process of operator learning adaptation is omitted,and the time of dexterous hand imitation task is reduced.In order to complete the task of imitating arm playing piano,UR5 robot arm is used to simulate the arm motion track to reach the target point,and then the dexterous hand playing piano task is completed by imitating human hands.