Prediction Of Human Action Trajectories In Human-robot Collaboration

The application scope of robot has been more and more extensive,and the human-robot cooperation with people to complete the task has been seen everywhere.In the traditional human-robot collaboration,in order to ensure safety,the human-robot interaction should be in a state of separation.In this state,the human-robot interaction dimension is single and the efficiency is low.In the new human-robot collaboration,robot and human can safely cooperate and share the same working space,complementing human's ability,giving full play to the advantages between robot and human,so as to improve the degree of automation.In this environment,it is particularly important for robots to make timely predictions of human movements.Using the given motion data to learn how to best combine a set of complementary prediction methods so as to actively assist people in their work is of great significance to improve the safety and efficiency of human-robot collaboration.In order to enhance human action prediction accuracy in human-robot collaboration,reduce the prediction error,this paper puts forward a method of prediction of human action trajectories with fusion skeleton coupling in human-robot collaboration.Using computer vision technology for the mainstream of non-contact recognition technology,deep neural network algorithm is adopted to establish the prediction model,and the coupling nature of human bones is used as the constraint conditions.The research is mainly carried out from the following aspects:First of all,to simplify the body geometry features in order to reduce the difficulties form geometry data for modeling human body.The open source algorithm Open Pose based on the technology of computer vision is used to extract the key points of the body and obtain human movement sequences,realize the mapping relationship between images and coordinates,deeply integrate images and coordinates,and represent the human action process through the change of coordinates.In the selection of action prediction model,each deep neural network is analyzed and compared.The Long Short-Term Memory(LSTM)Network is selected to establish the action prediction model.Secondly,the constraint conditions of human skeletal coupling were established by Laplacian Scoring(LS)algorithm.The distance relationship between key points in the video was scored.The lower the score,the stronger the coupling between the two key points.The key frames with strong coupling are combined to maintain the characteristics of data and group them to form skeletal coupling constraints.Using ISODATA(Iterative Self organizing Data Analysis Techniques Algorithm)clustering Algorithm to optimize the LS Algorithm.Making the corresponding "merge" or "split" of the key frame in order to remove the redundant data,keep the model's ability to understand the data,and enhance the authenticity of the data.This way could complete skeletal coupling constraints.Finally,a human action prediction model based on fused skeleton coupling LSTM network was established.Human movement sequences are sent to the LSTM network for training as training data.This goal is to predict the next 15 frames of movement sequences form the current 30 frames of movement sequences.In training,the constraint conditions of skeleton coupling are fused and the mapping relationship between current movement sequences and the future movement sequences are formed accurately.In the test,the new initial human movement sequences are sent to the trained action prediction model as the testing set,and then the future movement sequences are predicted in advance.In the comparison experiment,both of the LSTM that no skeleton coupling properties as constraint condition and the RNN(Recurrent Neural Network)do the contrast with the model in this paper.The comparison of experimental data shows that the accuracy of LSTM can be further improved while combine skeleton coupling properties as constraint condition,the accuracy of action trajectory prediction is more than 80%,which is higher than the unimproved LSTM and RNN.The action trajectories of the operator can be predicted more reasonably.The effectiveness and adaptability of the algorithm are verified by this comparison experiment so that human-robot collaboration can be basically implemented in a safe and efficient state.