Research On Real-time Spatiotemporal Detection Technology Of Workshop Personnel Production Behavior Based On Multiple Data Stream

The intelligent detection of production action in shop-floor is an important content of maintaining shop-floor safety,and it is one of the main technical bottlenecks to improve the degree of shop-floor intelligence.To solve the problems of low judgment reliability of real-life,poor accuracy of real-time action temporal and spatial positioning recognition,we proposed a real-time spatio-temporal detection method for shop-floor human production action based on multi-stream,which can ensure the reliability,real-time and accuracy of intelligent action detection in shop-floor.Firstly,preprocess each data stream,construct the digital expression vector of production action,and judge whether there is a real-life target in front of the station based on the real-time infrared image stream.When there is a real-life target,locate and classify the production action of these target in time domain based on the 30 frames human skeleton nodes sequence data stream at the current time point,and fully extract the temporal characteristics of production action and the internal correlation characteristics between each human joint nodes,quickly output the classification results of production action at this time point.When the classification result is a target action class,according to the temporal location classification result,the action spatial location recognition are carried out based on a single RGB image at this corresponding time point,the coding structure with multi-head attention mechanism is used to fully extract the target information,with the interactive instance matching mechanism of decoder,capture the semantic information of production action,so as to accurately output the category and target location of all production actions in this time point,and finally realize the reliable,real-time and accurate detection of production action.According to our method and combined with the actual requirements of the project cooperation unit,the real-time spatio-temporal detection software system of production action was designed and developed.It has passed various function and index check,and operated stably for more than half a year in the actual shop-floor scene,and obtained more than 80000 detection records,which verified the feasibility of this technical method.