Research On Efficient Hand Detection And Gesture Recognition Methods Based On Deep Learning

Hand detection and gesture recognition in images are crucial in the computer field,and are of great significance in many computer vision applications.Due to the variability of hand shapes and the clutter of the image background,accurate and fast hand detection and gesture recognition are still a challenging task.In order to improve the accuracy of hand detection and gesture recognition in complex and unconstrained scenes,and to address the problem slow detection speed,we propose two deep learning-based hand detection and gesture recognition methods.Evaluation experiments were conducted on three benchmark datasets,Oxford hand dataset,Ego Hands dataset,National University of Singapore(NUS)hand posture dataset and self-made dataset.The main research includes the following two aspects:(1)Aiming at the problem of low accuracy of hand detection and gesture recognition in complex and unconstrained scenes,we constructe a novel ResNet101 and fusion networkbased fully convolutional network RF-FCNet.The network utilizes ResNet101 as the hand feature extraction network,then the precision hand prediction fusion network is designed by fusing the deconvolution network and the residual structure to generate a multi-scale hand feature map,and realize hand detection and classification via a single convolution layer.Experimental results on the Oxford hand dataset show that the m AP of RF-FCNet reaches87.2%,which is superior to other mainstream models.Experimental results on the NUS hand posture dataset show that the m AP of RF-FCNet is as high as 99.2%,which is superior to other state-of-the-art models.Experimental results on three benchmark datasets show that the RF-FCNet can effectively improve the accuracy of hand detection and gesture recognition.(2)To address the problem that the detection speed of hand detection and gesture recognition methods is less researched or the detection speed is difficult to achieve real-time detection,on the basis of RF-FCNet,we propose a new SqueezeNet and fusion networkbased fully convolutional network SF-FCNet.The network uses the first 17 layers of the lightweight network SqueezeNet as the hand feature extraction network.Utilizing the characteristics of SqueezeNet that it loses less precision and greatly reduces model parameters to improve the detection speed while reaching a certain accuracy.Experimental results on the Oxford hand dataset show that SF-FCNet can achieve an accuracy of 84.1%and 32 FPS,which is superior to other state-of-the-art models,and FPS is almost 1.45 times faster than RF-FCNet.The experimental results on the NUS hand posture dataset show that SF-FCNet can achieve the same accuracy as RF-FCNet,and the FPS is almost 1.41 times faster than RF-FCNet,which proves that SF-FCNet can significantly improve the detection speed of hand detection and gesture recognition while ensuring a high accuracy,which has a certain versatility and effectiveness.The detection results on the self-made dataset also show that SF-FCNet has strong generalization ability and practicality.The experimental results on the benchmark dataset and the self-made dataset show that the propose RF-FCNet and SF-FCNet have excellent performance in accuracy and detection speed,and are suitable for two different tasks of hand detection and gesture recognition.This shows that the proposed architecture has better versatility.