Real-Time Dynamic Gesture Recognition Based On Deep Learning

With the popularization and development of computers,human-computer interaction tech-nology has become an indispensable part of people's daily lives.Due to its natural and convenient operation,the method of gesture interaction that does not require wearable devices has become an emerging human-computer interaction method.It can be widely used in scenarios such as game production,medical device operation,and multimedia device control.Therefore,with the matu-rity of deep learning technology in recent years,gesture recognition technology based on computer vision has become a hot research field in human-computer interaction.In order to explore more accurate convolutional neural networks,many studies have simply designed the network structure from the perspective of offline testing,increased the network scale and calculation scale,and ig-nored the feasibility of deploying the system on mobile platforms.In addition,in actual application scenarios,how to deal with the continuous and untrimmed input video stream and perform online and real-time recognition also increases the difficulty of gesture recognition to a certain extent.In view of the above deficiencies and difficulties,this paper builds a dynamic gesture dataset 101 Ges-ture that is closer to the actual human-computer interaction scene on the premise of ensuring the real-time nature of the system and the feasibility of deployment.A complete real-time dynamic gesture recognition system is implemented.The main work and innovative research results of this thesis are as follows:1.A dynamic gesture dataset 101 Gesture was constructed.Aiming at the difficulties of dy-namic gesture recognition and the shortcomings of the existing data sets,this paper starts from the actual application scenarios,and constructs a gesture data set 101 Gesture that is more close to the actual human-computer interaction scene,and has a richer variety of ac-tion instance lengths.The comparison shows the advantages and characteristics of the data set.2.Designed a lightweight gesture detection network MotionNet and recognition network Ac-tionNet.From the perspective of lightweight gesture recognition,gesture detection networks use spatial information in RGB images and time-series motion information in differential images to perform gesture instance detection in a continuously-input video stream.Gesture recognition networks use multi-level,multi-scale feature fusion to enable them to adapt to inconsistencies in the time scale of actions in actual scenes,ensure the accuracy and robust-ness of the network,and perform gesture classification.3.A complete real-time dynamic gesture recognition system is designed.Starting from the actual application scenario,based on the lightweight networks MotionNet and ActionNet,the buffer queue mechanism is combined,and gesture detectors and gesture classifiers are respectively designed and connected in series to output through filter post-processing.This method uses the detector as the "switch" of the classifier,which can enable the convolu-tional neural network to run online effectively by using the sliding window method,and save computing resources and storage resources to the greatest extent.