Research On Assembly Action Recognition Based On Convolutional Neural Network

Assembly action recognition is the basis for manual assembly monitoring,humanrobot collaboration,and ergonomic analysis of assembly operations.At present,mass customization has become a trend in the manufacturing industry.In the assembly process of mass customization products,the product structure is complex and the assembly steps are many.Once the assembly process is missing,wrong assembly or even the irregular operation of the workers will affect the product quality.Negative Effects.Therefore,it is necessary to carry out research on assembly behavior monitoring on the assembly line.For this purpose,the following studies were carried out:(1)A new network architecture based on attention spatiotemporal features is constructed,and an assembly action recognition method based on surface EMG signals and inertial signals is realized.The network model consists of data input layer,spatiotemporal feature extraction layer,attention module and fully connected classification layer.The spatiotemporal feature extraction layer is a residual module stacked sequence modeling structure that combines causal convolution and dilated convolution,and has the ability to extract time series features of assembly actions.Using the wearable sensor MYO armband,a dataset of EMG and inertial signals was established containing six types of assembly movements.Based on the assembly action dataset,the influence of different signal preprocessing methods and different normalization methods on the model is analyzed;the performance of the network model and CNN convolutional neural network,long-short-term memory network and twostream convolutional network are analyzed.(2)A workshop personnel assembly action recognition model based on the graph convolutional network model of attention mechanism and multi-scale feature fusion is proposed.The model innovatively applies graph convolutional network to the task of assembly action recognition and can extract assembly actions.The latent relationship features with assembly tools can improve the accuracy of assembly recognition.The attention mechanism module enhances the network’s ability to extract key information in the assembly behavior image,and the multi-scale feature fusion module enables the network to better extract image features at different scales.In order to test the effectiveness of the proposed workshop personnel assembly behavior recognition model,a data set containing 15 workshop production behaviors was constructed using the Kinect vision sensor,and experiments were carried out on this data set.The average assembly behavior recognition accuracy on the dataset reaches93.1%.(3)An assembly action recognition network model based on video frame motion excitation aggregation and temporal difference is proposed.The network model is mainly composed of motion excitation and aggregation module and temporal difference module.The backbone of the network model is based on the residual network Resnet101 network.,which is composed of three modules: motion excitation module,time aggregation module and time series difference module,which are used to capture short-term and long-term temporal features of continuous video frames,enhance the receptive field of the model,and improve the recognition accuracy of assembly tasks.This paper proposes an assembly action recognition method based on deep learning.Adding different network modules such as attention mechanism and graph convolution module to the assembly action recognition model improves the accuracy of assembly action recognition.It is of certain significance to realize the behavior control of assembly personnel and realize the intelligent monitoring of the production workshop.