Human-robot Collaborative Assembly Intention Recognition And Prediction Driven By Data+ Knowledge

At present,the social production demand is constantly developing in the direction of small batch and personalized customization,and traditional manual operations and teaching industrial robots are difficult to meet the flexible and efficient needs of intelligent manufacturing,especially assembly operations.As a new generation of robots working in collaboration with humans,collaborative robots need to be able to accurately identify and predict the operator’s intention,so as to cooperate with the operator efficiently,safely and naturally,and jointly complete the assembly task.Aiming at the human-robot collaborative assembly scenario,this paper adopts a data+knowledge-driven approach to study the recognition and prediction of the operator’s work intention by collaborative robots.The main research work is as follows:1.Construction of knowledge graph in assembly domain based on deep learning.For the operation scenario of human-robot collaborative assembly,the knowledge data in the assembly field was obtained through various channels,and the data was preprocessed to obtain the assembly corpus of 71219 words;The entity type and relationship type of the knowledge graph to be built are defined,the original graph pattern layer is established,and the label labeling task of the assembly corpus is carried out through the annotation software combined with the subsequent information extraction task.Firstly,the information extraction method is carried out by pipelined extraction,that is,named entity recognition is based on Bert-BiLSTM-CRF model,and then entity relationship extraction is carried out based on Bert-BiGRU-ATT model,and the F1-score of the two reaches 84.02%and 94.92%,respectively.Secondly,the knowledge processing of the extracted structured data,the merging and fusion of entities with the same meaning,and Neo4j is selected as the storage scheme for assembling the knowledge graph.Finally,the knowledge graph is displayed through a visual interactive interface to form a<operator>knowledge graph in the assembly domain centered on sum<assembly object>,containing 2724 triples.2.Operator intent recognition and prediction based on multi-source evidence fusion.Combined with the situation of the assembly site,the distance,angle,movement speed and assembly prior knowledge of the operator relative to the entity are selected as the elements of intention expression,and quantitative calculations are carried out to represent the elements of intention expression as the interaction probability function of each potential target at present,and the improved D-S evidence synthesis rule is used to fuse the four elements of intention expression to identify the assembly intent in the process of human-computer collaboration from multiple aspects.At the same time,the intention prediction is divided into the first prediction of intention and the intermediate prediction of intent,with the current assembly state determined,the next assembly task and the entities involved are predicted based on the relationships between different levels of entities in the knowledge graph.3.Operator assembly job intent recognition and prediction experiments.Design the experimental scheme,select the first-level teaching reducer as the assembly object,build the human-robot collaborative assembly experimental scene,firstly use the Kinect camera to collect the work site information,calculate the collected human skeleton point and entity coordinate data based on the method of D-S evidence synthesis theory,obtain the probability distribution of human assembly objects under different intention expression factors,and combine the prior knowledge of relevant assembly entities in the knowledge graph to recognize the operator’s intention in the current assembly scenario.Secondly,the current intention is input into the knowledge graph,and according to the process knowledge in the assembly knowledge graph,it is linked to the corresponding assembly entity object and higher-level assembly process,process,operation standard,etc.,to predict the operator’s next assembly operation intention.The experimental results show that the data+knowledge-driven method based on this paper can well complete the intention recognition and prediction of operators in the human-robot collaborative assembly scenario,and has good interpretability and applicability.