Research On Production Behavior Detection Based On Deep Learning In Factory Environment

To optimize the production process and human resource in the factory environment,it is common to manually record the production behavior of workers in the workshop.Compared with computer vision,human vision tends to be influenced by personal status,and it is difficult to make long-term observation.Therefore,an automated way to detect the activity temporally is needed.Currently,the development of deep learning and temporal action detection provides a new direction for production behavior detection in the factory environment.The purpose of temporal action detection is to detect both the action class and temporal boundaries from the untrimmed video.Hence,the temporal action detection technique can be used to detect the production behavior information automatically,resulting in reducing the labor expense.However,the diversity of the factory environment also has an impact on the production behavior detection.Considering the detection requirements of general and complex factory environment,the main contributions are as follows:(1)In general factory environment with simple scenes,to improve the training and detection speed of detection algorithm,an R-C3D-based fast temporal action detection iR-C3D algorithm is proposed.In our iR-C3D:1)for the efficiency problem at the stage of feature extraction,the ECO network is applied,instead of C3D network,to greatly improve the speed of the training and detection;2)the Non-Local attention mechanism is used to capture long-range dependencies in the spatio-temporal features;3)by optimizing the loss function,the more accurate temporal boundary information can be obtained.The simulation experiments on THUMOS’14 dataset shows that the performance improvement of iR-C3D is verified by reaching a 35.2%mAP@0.5 on THUMOS’14.Moreover,experiments are conducted in the general factory environment,the experimental results prove the effectiveness of iR-C3D in the general factory environment.(2)Considering the complex factory environment with excessive useless background,light changes and occlusion,a production behavior detection framework(FERM)is developed.The main contents include:1)for the large quantity of useless backgrounds and light changes,YOLO v3,which has the strong generalization ability and robustness,is used to segment the area where worker is in the foreground of the video,which can reduce the interference to detection caused by useless information;2)for the occlusion problem in the complex factory environment,a time threshold-based merge strategy is devised to merge the adjacent fragments.The experimental results show that FERM can effectively improve the detection accuracy in the complex factory environment.In terms of the deep learning and temporal action detection methods,this paper propose the novel production behavior detection algorithm.The simulation experiments prove the effectiveness of the proposed methods.Therefore,our research provides a basis for the optimization of production process and human resource.