Research And Application Of Gesture Recognition Algorithm Based On Deep Learning

With the development of social economy,life style is becoming more and more convenient.The human-computer interaction mode is an indispensable part of life.The human-computer interaction has developed from the keyboard and mouse,and then to handwriting and voice control,etc.Consequently,the interaction mode is gradually becoming more and more convenient.Hand gesture is one of the human body language,which has the characteristics of being intuitive,flexible and rich information.It is an indispensable part of the interpersonal communication.In recent years,computer vision-based gesture recognition has become a hotspot in scientific research and application.The corresponding mathematical algorithms are used to identify human gesture language to achieve human-computer interaction.The dissertation is organized as follows:(1)In order to solve the problem of the influence of the change of illumination intensity on the accuracy of gesture recognition in different recognition environment,an improved Faster RCNN hand gesture recognition algorithm is proposed based on optimized Res Net-50 network in this work.Compared with the ordinary Faster R-CNN algorithm,the proposed algorithm used with Res Net-50 network,improves the feature learning ability of the network.Furthermore,using the IBN(instance batch standardization)optimize the Res Net-50 to the feature learning for character content,and the different recognition environment.The experiment shows that the proposed algorithm achieves the recognition rate of 98.7% on test set,with higher effectiveness and robustness compared with state-of-the-art hand gesture recognition algorithm.(2)In order to solve the problem that the mobile terminal detects the gesture speed too slowly,the improved Yolo V3 gesture recognition algorithm based on Mobile NetV2 is proposed.The anchor boxes are clustered by Kmeans algorithm to obtain anchor boxes number K=9.Using the lightweight Moblie NetV2 network to extract features from the dataset,the Moblie NetV2 network has a small amount of computation,the amount of parameters that the model has after learning is only 0.1 times that of Resnet-50.Finally,multi-scale stitching detection and recognition is implemented through the Yolo V3 network.The experimental results show that the average recognition rate of YoloV3-MobileNetV2 is 92.7%,the training learning time is greatly reduced,and the convergence speed is faster.In the hardware environment of this paper,the FPS is 33,which meets the real-time detection requirements of mobile terminals.(3)Based on the construction of YoloV3-MobileNetV2 algorithm,the paper designs a contactless gesture typing system,which is a practical application of human-computer interaction mode based on gesture recognition.The contactless gesture typing system design is divided into five stages: problem definition and planning,requirements analysis,software design,program coding,and software testing.The design of program coding was mainly based on Py Qt5 and Py Charm,and the design basis of gesture module was the gesture recognition algorithm proposed in this paper.The system was designed to enable simple typing with gestures.