Research On Hand Gesture Recognition Algorithm Based On Convolutional Neural Network

Recently,as one of the most direct and convenient Human-Computer Interaction(HCI)methods,hand gesture recognition has attracted lots of attention in various fields.Furthermore,it plays an important role in many fields such as virtual reality,assisted vehicle control system,sign language recognition and personal wearable system.Traditional gesture recognition technology usually uses sensors or computer vision to obtain the gesture data,and then manually designs the specific gesture feature to train the classifier to recognize hand gesture.However,traditional gesture recognition methods mainly have these following defects: First,the device is expensive and inconvenient to carry.The gesture recognition method based on sensors has high requirements on the device and requires the user to wear extra equipment,which limit the promotion of this method.Second,most of the researches on gesture recognition based on computer vision focus on normal illumination,ignoring the condition of abnormal illumination such as intense illumination and weak illumination,resulting in poor robustness to illumination.Third,these methods need to design gesture features artificially,which is very subjective and limited for these features are designed by personal experience.Thus the system is of poor learning ability and has a relatively low recognition rate,which can only recognize specific gestures.To solve these problems,this thesis proposes a new gesture recognition method based on convolution neural network.The main contents of the thesis are listed as follows:1.This thesis focuses on the application of Multiple Dimensional Convolutional Neural Network(MD-CNN)in video-based dynamic gesture recognition system and proposes a high-precision gesture recognition subsystem.First of all,a large amount of gesture video data are collected via ordinary camera and represented as ordered image sequences.Then these image sequences representing the corresponding gestures are input to the MD-CNN to perform feature extraction and gesture classification.In particular,for the specific problem of gesture recognition,a series of improved algorithms for the network are proposed,including batch normalization algorithm,moving average algorithm and adaptive moment estimation algorithm.Experimental results show that the improved network converges faster with a higher recognition rate.2.In order to improve the system's robustness to illumination,this thesis proposes a gesture recognition method based on Channel State Information(CSI)by using Two Dimensional Convolutional Neural Network(2D-CNN),thus a robust gesture recognition subsystem is constructed.First,by analyzing the CSI signal,this thesis proves the correlations between the change of the dynamic path length and the amplitude of the CSI signal theoretically and experimentally.Next a series of preprocessing operations are performed to original CSI signal,mainly including wavelet denoising operation.After that,the amplitude of the denoised CSI signal is converted into an image and is input to the 2D-CNN to extract features and classify gesture.3.This thesis proposes a high-precision and robust gesture recognition system based on multi-source data fusion method.Firstly,the weight coefficients of these two gesture recognition subsystems are determined according to the validation dataset in the offline phase.Then the weighted fusion method is performed on the classification results of these two subsystems in the online phase to obtain the final gesture class.The results show that this method can effectively improve the system's robustness to illumination and the accuracy of gesture recognition.