A Robot Trajectory Programming-by-demonstration System Based On A Pen-shape Marker

With the continuous expansion of the industrial robot market,the application fields of industrial robots have also expanded greatly with the needs of the transformation and upgrading of the manufacturing industry.In the field of emerging industrial robots application scenario such as welding,laser cutting and spraying,traditional robot trajectory programming methods,such as teach pendant and kinesthetic programming,have the disadvantages of high technical threshold and low programming efficiency,which cannot meet the demand for small batch and short-cycle production.This has become a threshold for further application and promotion of industrial robots.Aiming at the problem of traditional industrial robot trajectory programming methods,this paper designs and develops a robot trajectory programming by demonstration system based on a pen-shape marker.Efficient trajectory programming teaching is achieved through an innovatively designed teaching pen and machine vision algorithms.The system is developed based on the Robot Operating System platform,and is equipped with a graphical interactive interface and a gesture interaction interface,which is well scalable and user-friendly.The specific work content can be summarized as follows:Design and development of the trajectory tracking module.According to the requirements of trajectory-programming task,a simple and low-cost pen-shape marker was designed innovatively,and a teaching plan based on machine vision to capture the trajectory of the pen was proposed.By the use of Kinect sensor,the developed system collect the point cloud data of the robot workspace.Then,the designed algorithm extracts the pose information of the pen from the point cloud data.After the camera calibration,the pose information of the marker in the robot base frame is obtained,and the programming trajectory is visualized by the rviz tool provided by ROS.Design and development of the robot motion planning and control module.Design the control strategy of the system to meet the needs of the trajectory-programming task.Through continuous sampling of the posture of the marker,the track points on the demonstrated trajectory is obtained.By the use of Move It package provided by ROS,a complete robot trajectory can be planned and the demonstrated track can be reproduced by the robot immediately.A graphical user interface for the system is designed and developed through multi-thread programming method,in witch,the ROS node is encapsulated in the thread of the Qt process.Design and development of human-machine interaction module.In order to furtherimprove the friendliness of the system,a gesture recognition interaction module based on image recognition was designed and developed.According to the system requirements,7 control gestures were designed and the gesture data set was collected.The improved Let Net-5 network is trained using the data set,and the trained network model is integrated with the system to implement gesture interaction.System function verification and performance evaluation experiments.The experiment is based on the drag teach track programming function provided by the UR robot's CB3 controller,and the system performance is evaluated from two aspects:programming efficiency and programming accuracy.