Research On Human Robot Interaction Based On Stoke Information For Manipulation Tasks

The development of human-robot interaction technology can adapt the robot to the increasingly complex and diverse working environment and work tasks.Therefore,it is of great significance to explore new human-robot interaction methods and simplify the human-robot interaction workflow.Stroke interaction is a kind of interactive method with simple drawing and stable recognition.It has been widely used in home computers,mobile phones,tablets and other devices.In the field of human-robot interaction,strokes have been applied to robot locomotion tasks,but for manipulation tasks,the application of strokes still has a lot of room for improvement in terms of information transmission efficiency and control of robot operation.Aiming at the above problems,this paper introduces three types of stroke information,such as stroke point,stroke track and stroke vector,to solve the interaction problem in robot task,and proposes the human-robot interaction method of stroke information.This method only uses stroke information as the instruction transmission medium.Combining image processing,pattern recognition,path planning,force speed control and other technologies,the interactive extraction of object parameters and the interactive control of the manipulation process are simplified,the interaction process is simplified,and the research direction of human-robot interaction methods based on stroke information is expanded.It provides new solutions and research ideas for the selection of human-robot interaction methods in complex manipulation tasks.Firstly,an interactive extraction method of task parameters based on stroke point information is proposed.This method uses image registration technology to realize one-to-one correspondence between color images and depth image pixels,and solves the projection of stroke points drawn by the operator into three-dimensional space.The solution provides a basis for the subsequent projection of the stroke gestures.In addition,for the case of color images,a method using multi-view color images is proposed to realize projection of the stroke point to the 3D space.This scheme uses the kenelized correlation filter tracking technology to achieve multi-view image acquisition,and designs a variety of image frame sampling strategies to improve the accuracy of the solution of the spatial position.Considering the tasks of opening doors and twisting valves,the transformation process from stroke point to working object parameters is expounded,and the effectiveness of the proposed method is verified by experiments.Secondly,aiming at the problem that the amount of instruction information of the stroke point is small,the method of extracting the object parameters by the continuous stroke trajectory is proposed.The innovation of this method is that the process of directly identifying the object is transformed into a stable process of recognizing the stroke gestures;in addition,the method is not limited to identifying the type of the work object,and can further utilize the feature of the three-dimensional space point corresponding to the stroke gestures to extract the position and size of the objects to make full use of the characteristics that the stroke gestures carry a large amount of information,and the extraction of the parameters of the objects is conveniently performed by one-time drawing.The parameters extraction test for the typical disaster operation tasks of the robot was carried out,and the effectiveness of the proposed method to solve the problem of parameter extraction of the objects was verified,which laid a foundation for the interactive control of the subsequent operation process.Then,for the problem that the training of the stroke recognizer needs a large number of samples,an iterative synthesis method of stroke gestures is proposed.Most of the traditional synthetic methods only evaluate the quality of synthetic strokes.On the basis of evaluating the quality of strokes,the design method of this paper feeds back the evaluation results to the generation process,and iteratively generates a balanced dataset of strokes.The method uses Bezier curve to construct the stroke gesture template,which solves the problem of template construction for single stroke and multi-stroke.The stroke gesture synthesis experiment was carried out on the public dataset.The experimental results show that the dataset generated by the method can simulate the operators' manually drawing dataset and the mthod realizes the low-cost acquisition of the stroke gestures.Finally,in order to solve the interactive control problem during the execution of the task,a robot operation process interactive control method based on stroke vector information is proposed.The innovation and implementation difficulty of this method is how to transform the stroke vector information into the robot control signals.In this paper,the stroke vector is used as the source of expected force and expected speed.With the help of force-speed hybrid control and virtual fixtures,the transformation process of stroke vector to robot control signals is realized,and the motion of the end-effector and the external forces are controlled within reasonable limits.The test platform was built.Taking four typical disaster tasks as examples,combined with the parameters obtained in the parameter extraction process,the motion trajectory was generated.Using the real-time input stroke vector,the operator is able to carry out the real-time intervention of the operation process.The feasibility of the proposed interactive control method is verified,and the flexibility of the method in guiding the robot operation is demonstrated.