Research And Application Of Human Motion Trajectory Prediction For Human-Robot Collaboration

At present,the research of human-machine collaborative assembly is very hot in the field of intelligent manufacturing.Humans and robots working together in a shared workspace can give full play to their respective advantages and significantly improve productivity.The main challenge in human-machine collaboration lies in the randomness of human behavior,motion trajectory and assembly sequence.In order to accurately predict human intentions,many researchers use prediction algorithms to process multiple modal information at the same time.For example,in assembly tasks,human hands and heads rotate simultaneously in order to accurately grasp parts.Considering these two modal information is necessary to predict human intention algorithms.In addition,in order to study the real-time and robustness of human-machine collaboration model,most researchers choose to conduct experiments in ROS environment.However,installing and debugging ROS systems adds a lot of burden to researchers;Moreover,the incompatibility between the system and the fast-updating programming language and deep learning library makes it difficult for the ROs-based human-machine collaboration model to iterate with technological updates.Aiming at the above problems,a multi-modal hand trajectory prediction algorithm and a crossplatform communication system for human-machine collaboration are proposed.Through the modeling of small wooden chair assembly task,the physical experiment of humanmachine collaborative chair assembly was carried out.The details are as follows:(1)Human-machine collaboration experiment platform development.The platform consists of a robotic arm,a two-finger electric grip and a camera.In order to coordinate various hardware and software in human-machine collaboration and reduce deployment cost,a loose-coupling human-machine collaboration system was designed.The system is a centralized network,which abstracts hardware programs and visual algorithms into independent nodes.Nodes communicate by subscribing/publishing messages.Researchers can perform experiments by simply accessing the lab’s LAN.The usability and real-time performance of the human-machine collaboration experimental platform were tested through the grab-andplace experiment of visual positioning,and the results show that the experimental platform can accomplish the basic tasks of human-machine collaboration.(2)Multi-modal human trajectory prediction algorithm.In this algorithm,human hand trajectory mode and face orientation information are input into LSTM to predict human hand trajectory.First,two subjects were invited to build a data set in a real human-computer collaboration environment.SG filtering,cubic spline interpolation and remote cross-sampling were used to smooth and enhance the human hand trajectory.Support vector machine(SVM)is used to classify face orientation modes.In the end,the two modes will undergo direct feature fusion.LSTM and recurrent neural network(RNN)were trained using selfbuilt data sets.The performance of multi-mode model and single-mode model,LSTM and RNN were compared under different proportions of observable action sequences.The results show that multi-mode LSTM has certain advantages.(3)Application of human trajectory prediction in human-machine collaborative assembly scenarios.Task hierarchy model and Markov model were used to model the collaborative assembly task of small wooden chair.Meanwhile,a template matching algorithm based on anisotropic filtering is proposed to sense assembly state.Based on the man-machine collaboration platform,a physical experiment was carried out on the wooden chair,and the multi-modal human trajectory prediction algorithm was applied in the experiment.The experimental results show that the robot has the ability of active cooperation and the cooperative action is smooth.