Research And Application Of Gesture Recognition Algorithm Based On Deep Learning

With the development of science and technology,human-computer interaction is more and more widely used in daily life,bringing new experiences to people’s life.With rich semantic information and high flexibility,hand gestures have an important position in human-computer interaction,so how to recognize hand gestures quickly and accurately is a research topic attracting much attention.The traditional gesture recognition methods have some problems,such as low robustness and difficult deployment.Therefore,aiming at the problems existing in traditional methods,this thesis proposed a gesture recognition algorithm based on lightweight YOLOv4 and applied it in combination with the characteristics of gestures.The main work contents were as follows:(1)Production of data sets.In this thesis,2120 photos were screened from the public data set,which were divided into 16 kinds of gestures and manually labeled.In order to prevent overfitting of the network model and improve the generalization of the network model,data enhancement techniques such as random scaling and flipping were used to expand the original gesture data set to 12720 pieces.(2)Improvement of YOLOv4 algorithm.Aiming at the problems of YOLOv4 algorithm with large number of network model parameters and low detection speed,this thesis proposed a lightweight object detection algorithm N-YOLOv4 for gesture recognition.Firstly,in order to accelerate the training speed of the model,the idea of transfer learning was introduced in the training,and the influence of different pre-training weights on the performance of the model was explored,from which the pre-training weight file with the highest accuracy was selected.Secondly,GhostNet was used to replace the backbone feature extraction network of YOLOv4 and deep separable convolution was used to replace the 3×3 conventional convolution in the non-backbone feature extraction network of YOLOv4,in order to reduce the number of parameters of the model and improve the detection speed of the model.Then,the receptive field of the network was expanded by replacing part of the 1×1 conventional convolution with CSC module.Finally,the PANet network structure of YOLOv4 was improved by using residual edges and quintic convolution module to enhance the extraction of features by the network and improve the detection accuracy of the model.Compared with the original YOLOv4,the experimental results show that when the detection accuracy is only slightly affected,the weight file size decreases by 76.72% and the FPS increases by 10.85%.(3)Construction of gesture recognition system.In order to accelerate the application and popularization of gesture recognition in daily life,a simple gesture recognition system was designed based on N-YOLOv4 algorithm and PyQt5 and other technologies.The system required proper login to access the main system screen.The detection module of the system used the n-YOLOv4 algorithm,and the interface was built using PyQt5.The functional modules of the system included picture detection,video detection,music playback,etc.The core function is human-computer interaction,which captured gestures through the camera for recognition to control the movement of the warplane model in the game.The experimental results show that the system is practically feasible.