Research On Gesture Recognition Based On Deep Convolutional Neural Network

Gesture recognition is an important direction of research of human-computer interaction technology, which has broad application prospects, such as medical surgery, somatosensory game and sign language recognition. Gesture recognition is usually divided into two parts:gesture detection and gesture recognition, the challenges are respectively originated from illumination changes, occlusion, complex background, interference of likelihood skin color and the diversity and ambiguity of the gesture, and image quality.Without manual definition or feature selection, convolutional neural network, regarding images as input directly, can extract multi-scale features via automatic learning ability, fault tolerance and parallel processing ability. In this dissertation, we explore the gesture image processing and recognition method based on deep convolutional neural network focusing on American Sign Language data set. The main contents and innovative work include:(1) The current research status of convolutional neural network and gesture recognition are summarized. Next, the concept, characteristics and common network structure of convolutional neural network are analyzed. At the same time, the conception, characteristics and acquisition methods of the depth image are described. Finally, we discuss the related knowledge and theories of statistics learning, optimal classification hyperplane and support vectors.(2) In order to suppress the interference from skin color or complex background, a new method on gesture segmentation based on depth information is proposed in view of the characteristics of American Sign Language data set. First, the method converts every depth image to grayscaled depth image, and the corresponding mask image is obtained via the depth differences between each pixel and gesture pixel base and morphological close operation; and then the regions of interest are segmented using the logic "and" operation between the mask image and the luminance image deduced from the corresponding RGB image. The experimental results show that the proposed segmentation preprocessing method can effectively segment the region of interest, which effectively suppresses the influence of non-hand pixels.(3) An efficient gesture recognition method based on deep convolution neural network (CNN-SVM) is proposed. In this method, multi-scale feature of the image is extracted by the convolutional neural network algorithm. Then, a CNN-SVM model is established by combining a convolutional neural network with a support vector machine. Finally, the gesture is recognized effectively after a large number of regions of interest images were trained. Experimental results show that the proposed method achieved the accuracy of 96.1%, is faster and more efficient than some existing methods like HSF+ RDF, SIFT+PLS, MPC, SAE+PCA and CNN.