Research On Static Gesture Recognition Based On Multi-Feature Fusion

The traditional gesture recognition method relies on the complex artificial feature extraction algorithm to extract the features of the image.The method has high requirements on the acquisition equipment,the background and gesture of human.With the powerful feature extraction ability,convolutional neural network is widely used in the feature extraction of gesture image.Feature extraction based on convolutional neural network can directly take the whole gesture image as the input of network.The deep hierarchical feature of the image can be extracted by convolution layers within the network,and the extracted features can more comprehensively describe the information of gesture image.However,the feature extraction method based on single convolutional neural network may have the problem of feature omission in gesture image,and the method can only extract deep hierarchical features of gesture image.The significant role of low hierarchical features(local features)in gesture recognition is neglected.In order to solve the above problems,the feature extraction method based on double-channel convolutional neural network is firstly studies in the dissertation.Based on the method,a static gesture recognition method based on multi-feature fusion is proposed.The major research work completed is as follows(1)In order to solve the segmentation problem of multi-skin color gesture image,the method of gesture image segmentation is designed based on multi-factors.YCbCr color model is used to segment the gesture image,then the segmented image is denoised by median filtering and flood filling algorithm.The method based on maximum connected area and centroid location is used to exclude the skin-like area and the skin-like area except the hand area(2)In order to solve the problem of insufficient feature information extracted from gesture image by single convolutional neural network,a feature extraction method based on double-channel convolutional neural network is given.Batch normalization(BN)layer is added to the VGG network for feature extraction.The gesture grayscale image is used as the input of the neural network to reduce the computational load of the network model.The optimized VGG network and the AlexNet network are used to extract the deep hierarchical features of the input image respectively.(3)A static gesture recognition method based on multi-feature fusion is proposed to solve the simplification and omission of the features.The original gesture image is segmented by a multi-factor segmentation method,and the binary image containing only the hand region is extracted,and then the local feature of the binary image is extracted,including Fourier descriptor and Hu moment.The double-channel convolutional neural network is used to extract the deep hierarchical features of the gesture grayscale image.Finally,the local features and the deep hierarchical feature after fusion are fused by cascade method.Softmax is used to classify and predict gesture images.