Vision-based Gesture Recognition Research

With the high-speed development of artificial intelligence,the interaction between people and computers becomes increasingly intelligent.Exploring a simpler and more efficient human-computer interaction method has become a research hotspot.Gesture interaction as a kind of interaction method most commonly used by people has the characteristics of nature and good user experience,so gesture recognition technology has high research value.The gesture recognition based on computer vision is an economical,simple and non-contact interaction method.By using only a common monocular camera,the interaction between human and computer can be realized.Therefore,gesture recognition based on computer vision has become the main research direction currently.Since each person behaves differently,this may lead to ambiguity,when different people express the same gesture,which will affect the recognition of gestures.Therefore,it is especially important to extract appropriate and robust gesture features.In this paper,the algorithms of gesture image processing,static gesture recognition and dynamic gesture recognition are analyzed and studied.The main work of this paper is as follows:(1)In the gesture image segmentation stage,it is necessary to segment the gesture partial area from the background.Firstly,we have proposed a gesture segmentation method based on skin color model fusion.This method can get a better segmentation effect.The method firstly separates the gesture regions based on the HSV and YCrCb color space models,and then performs the AND operation on their result images.Finally,the mathematical morphology is used to eliminate the noise interference,and the maximum connected region method is used to eliminate the skin-like noise regions.Thus a complete gesture has been extracted.(2)For static gesture recognition,because traditional manual design features are time-consuming and labor-intensive,and artificial design features are relatively simple,there is great subjectivity and complexity,and it is difficult to extract robust gesture features.Therefore,we have used a convolutional neural network in deep learning to design a convolutional neural network based on gesture binary image which is obtained by using the method in(1).The use of gesture binary images can reduce the influence of the background in the image,thereby highlighting the features of the gesture.The data augmentation is used to make the model have certain rotation,translation and scale invariance.(3)Aiming at dynamic gesture recognition,we have proposed a method based on sparse sampling and three-dimensional convolutional neural network.Because the traditional dynamic gesture recognition method involves detecting hand shape,designing gesture features,tracking gestures,etc.,this process is very complicated.The use of three-dimensional convolutional neural networks can automatically extract the temporal and spatial characteristics of dynamic gestures in a video stream,thus avoiding the complexity of traditional methods.And the original gesture video is sparsely sampled,so that the image frames input into the network are more representative and can reduce the computational complexity.Finally,the sparse sampling model fusion method has been used to further improve the accuracy of dynamic gestures.