Research On Workflow Recognition In Complex Factory Environment

The workflow recognition technology can use to compute and process monitoring video in production workshops for achieving detection and identification of manufacturing process quickly and accurately,which will greatly help industrial enterprises to standardize their production tasks and optimize the scheduling of manufacturing process.Different from the existing workflow technology,workflow recognition technology relies heavily on the existing computer vision technology.Firstly,potential task sequences in video are excavated through computer vision technology,and then corresponding workflow will be reconstructed by workflow modeling or comparison.However,due to the fact that the actual production environment is relatively complex(frequent light changes,and objects are blocked from each other),traditional motion recognition methods that rely on the detection and tracking of target objects cannot achieve effective recognition rates and are difficult to apply to complex production environment.In order to achieve effective workflow recognition,this paper will discuss two novel algorithm frameworks proposed through machine learning,biological algorithms and other ideas:(1)Workflow Recognition Method Based on 3D-CNN(3D Convolutional Neural Network).In order to extract the spatiotemporal points of interest in the video,that is,moving targets,we apply an adaptive threshold three-frame difference method to segment the motion regions,thereby reducing the complexity of the subsequent model training.Then,in order to accurately classify workflow tasks in the factory environment,we use a three-dimensional convolutional neural network combine with multiple views to train a classifier with high robustness.The advantage of this method is that the use of multi-view fusion greatly reduces the impact of object occlusion,light variation and other factors on the final recognition.At the same time,the neural network model does not require complicated feature extraction of the input stream,which improves the recognition efficiency.(2)An Automatic Workflow Identification Method Based on GA-3D-CNN(Genetic Algorithm-Three-Dimensional Convolutional Neural Network).In order to effectively identify the workflow sequence of the factory environment,we first use a state-based video stream segmentation method to automatically divide the input stream into individual tasks,and then use a trained three-dimensional convolutional neural network to classify the segmentation results.Then,a genetic algorithm with prior knowledge is used for permuting all possible workflow sequences,and the above classification results are input into a predefined objective function to calculate the final workflow identification result.The advantage of this framework is the use of global-wide workflow sequence recognition,which reduces the impact of classifier errors on the final recognition results to a certain extent,and also avoids the unnecessary performance consumption of exhaustive workflow sequences.The calculation model and the corresponding algorithm along with the systematic comparative experiments based on the dataset of real-life industrial master videos are given respectively to these two kinds of frameworks in the paper.Through experiments,we find that the two methods proposed in this paper are robust enough to the recognition of workflow in the complex environment such as factories,which shows high recognition rate and strong performance.