Research On Action Recognition And Operation Normative Discrimination Method In Manual Assembly Process

The human-machine collaboration will become the main form of social activities such as smart life and industrial production in the future.To realize human-machine collaborative operation in the manufacturing environment,the primary task is to enable the machine to recognize human operation actions and understand human work intentions,so that the machine can make effective decisions when assisting human operations.In this thesis,the assembly operation of mechanical parts is taken as the object.Based on the deep learning method,the action recognition and operation normative discrimination methods in the assembly process are studied,which lays an important technical foundation and theoretical basis for human-machine cooperation,and has important application value for promoting the intelligent transformation of enterprises.(1)The characteristics of the manual assembly process,which is dominated by experience and difficult to monitor,are analyzed,and a framework of action recognition and normative discrimination of assembly operation based on deep learning and knowledge management is constructed.The research method flow of key links including semantic structured modeling of the assembly process,assembly action segmentation and recognition,and job normative evaluation is given.Combined with knowledge graphs,machine vision,similarity analysis,and other technologies,the technical difficulties in each key problem are analyzed in detail,which provides a research approach for further research on assembly action recognition.(2)The discrete and unstructured description characteristics of assembly process information are analyzed,and a semantic information model of process knowledge for the assembly process is constructed to solve the problem that the process knowledge file is difficult to be formally expressed.Based on the analysis of assembly process information,the basic units of the assembly process and their relationship are extracted.The ontology information structure description method is used to complete the construction from process elements to the knowledge graph model.The process knowledge graph constructed can be used as a computer knowledge base,which is convenient for computers to retrieve,analyze and use,and provides a basis for intelligent evaluation of assembly process standardization.(3)The characteristics of continuity,similarity,and relevance of actions in the assembly process are analyzed,and an action segmentation model TSM-ResNet50+MS-TCN based on time shift module(TSM)and multi-stage time convolution(MS-TCN)is constructed.The time-lapse residual network(TSM-ResNet50)is built to extract the spatiotemporal features of assembly actions in the video.The fusion of time-lapse and residual modules reduces the number of parameters to be optimized and the number of network calculations while ensuring accuracy.The MS-TCN network is built as an action segmentation module.Based on the combination of dilated convolution and multi-stage network,the recognition and indirect segmentation of longterm assembly video actions are realized.The model can be directly used to process unedited assembly process video,avoid a large amount of video data processing work,and greatly improve the efficiency of assembly action recognition.(4)The possible deviation between the actual assembly process and the standard process caused by the flexibility of the personnel assembly process is analyzed.Combined with the action recognition results and the semantic information model of the assembly process,a normative discrimination method of assembly operation based on template matching is constructed.Based on the differential calculation and sliding window method,the abnormal data in the action recognition results are processed to reduce the influence on the extraction results of the actual assembly operation sequence.Based on the graph structure data search method,a method of extracting standard assembly operation sequence from the process knowledge graph based on the operation sequence of action recognition is designed.Based on the graph structure data representation method,the operation sequence feature matrix is constructed,and the similarity analysis between the action recognition operation sequence and the standard operation sequence is carried out to determine the standard sequence with the highest similarity,and then the normative discrimination of the assembly operation action sequence is completed.