Research On Human Motion Prediction For Human-machine Collaboration

In the process of man-machine cooperative assembly,the following problems mainly exist: because of the complex and changeable assembly action and the small movements,the assembly tasks performed by the workers are difficult to be accurately understood by the robot;in the man-machine cooperative assembly process,the order of different workers performing the assembly task is so unfixed that the robot task needs to be adjusted dynamically in real time;The inappropriate behavior of the robot in the process of man-machine cooperative assembly will cause potential safety risks to the workers,and the safety problem is the top priority.In order to solve the above problems,the human motion prediction method is studied for the accuracy of human motion prediction and the safety of human-machine collaboration.Firstly,the assembly behavior data set was established to extract the movement data and obtain the movement characteristics of the workers.Secondly,an assembly behavior identification method based on scene understanding is adopted to identify the tasks and task boundaries currently performed by workers.Then,combining the assembly behavior identification results,a prediction model based on the hierarchical Transformer encoder was developed for worker movement prediction.Considering the uncertainty of the predicted worker movement sequence,in order to fully utilize the collected worker movement information,we further use a hierarchical pre-training scheme into a hierarchical Transformer-based skeletal sequence encoder to explicitly capture spatial,short-and long-term time dependence at frame,segment,and video-level,respectively,reducing the error of the hierarchical-based Transformer encoder network prediction.Finally,the performance of the proposed algorithm model for identifying and predicting worker actions is tested in a large motion dataset and the produced assembly behavior dataset.Experiments show that the proposed method greatly improves short-term prediction accuracy in human-machine collaboration scenarios,and in long-term prediction,unlike the posture predicted by other prediction algorithms tends to be static or unnatural,the worker posture predicted by the proposed worker prediction model is effective.