Research On Abnormal Action Recognition Of Miner Based On Deep Learning

Coal mine safety production is closely related to the country’s economic development,and coal mine safety accidents are mostly caused by miners’ irregular operations and abnormal action,so it is important to study the abnormal action recognition of mine personnel to ensure coal mine safety production.At present,human action recognition research is mostly based on single-modal methods,which fails to effectively utilize the advantages of human action recognition multimodal data,especially in special scenarios such as coal mines where human action recognition research has the problems of insufficient feature information extraction and poor classification recognition accuracy.In order to make full use of multimodal data features,effectively improve the performance of miners’ abnormal action recognition,and at the same time build a lightweight network to ensure the practical application of the model,this thesis carries out research on miners’ abnormal action recognition based on deep learning on the basis of in-depth analysis and research of domestic and foreign action recognition results,and the related work is as follows.(1)A segmentation clustering optimization based inter-frame difference video key frame extraction method is proposed.To address the negative impact of redundant information of image video sequences on action recognition research,the original video sequences are segmented,then each video stream is clustered using K-means clustering method,and finally the key frames are extracted from the clustered video sequences using inter-frame difference method.The method can effectively remove irrelevant information redundant with behavioral actions and establish the foundation for subsequent feature extraction.(2)A miner abnormal action recognition method based on multimodal fusion and attention mechanism is proposed.To address the problem of insufficient information extracted from single-modal actional features,the spatio-temporal features of actional actions are fully extracted and the classification probability distribution is predicted by taking advantage of multimodal data and applying the principle of attention mechanism,and finally the maximum fusion decision is made.Through experiments,it is verified that the method has a high accuracy rate of action recognition.The laboratory dark light condition simulation dataset and coal mine scene abnormal action dataset are also constructed and expanded to lay the foundation for effective training of the benchmark model.(3)A lightweight multimodal fusion miner abnormal action recognition method based on knowledge distillation is proposed.The teacher-student network model of skeletal nodes and RGB images is built using the knowledge distillation learning method,and the accuracy and lightness of the model are further ensured by replacing the light-weight convolutional backbone network and adding the channel attention mechanism method.Through experimental verification,the method can achieve the desired results.Finally,a framework of miners’ abnormal action recognition system with fused target detection is also built to provide a reference for the research aspect of personnel action recognition in the intelligent construction of coal mines.This thesis has 62 charts,23 tables and 86 references.