Research On Hand Gesture Recognition Algorithm Based On Vision

Gesture,as a human-computer interaction mode,has better comfort in human-computer interaction than traditional keyboard and mouse.It has been widely used in smart home,robot control,sign language recognition,etc.,and has become a research hotspot in the field of human-computer interaction.Compared with gesture recognition based on data glove,vision-based gesture recognition only needs a camera to complete human-computer interaction tasks,does not need to wear contact data gloves,does not give people a sense of restraint,has the advantages of natural,simple and convenient interaction,and has become the mainstream method of gesture recognition.Vision-based gesture recognition technology uses single or multiple cameras to collect gesture information and uses specific methods to implement gesture recognition.According to different recognition objects,it can be divided into static hand gesture recognition and dynamic hand gesture recognition.In reality,the background environment of gesture interaction is complex and changeable,and different people may express the same gesture differently,which make it difficult for accurate gesture recognition and affect the user experience.In order to improve the user experience and build a robust gesture recognition system under complex background,this paper researches vision-based gesture recognition methods.The main contributions of this paper are as follows:(1)Method for static hand gesture segmentationGesture segmentation is the first step in the process of static hand gesture recognition based on vision.The effect of segmentation affects the later gesture classification results.In this paper,the depth information provided by Kinect camera is combined with skin color information of the hand to solve the problem that it is difficult to accurately separate the human hand from the complex background in monocular static hand gesture recognition.For the redundant arm area after hand segmentation,this paper innovatively uses distance transformation operation and palm segmentation circle to accurately and quickly remove the arm area.Experimental results show that this operation can improve the classification accuracy of the gesture classifier.(2)Method for static hand gesture recognitionThe features of the gesture binary image obtained by static hand gesture segmentation are extracted for gesture classification,this paper explores the classification performance of gesture classifiers under different feature extraction methods.Features are manually extracted and input to the support vector machine for training,this paper implements a traditional static hand gesture classifier.Convolutional neural network is used for automatic feature extraction and classification on binary images of gestures,the gesture classifier constructed in this paper has better recognition effect.(3)Method for dynamic hand gesture recognitionThis paper uses 3D convolutional neural network and convolutional LSTM network to capture the spatiotemporal features of video image sequences,and uses SPP network to extract local and global features.Finally,they are input into a fully connected network to achieve high-accuracy dynamic hand gesture recognition.Multi-modal video image sequences are used as the input of dynamic hand gesture classifiers.In this paper,two kinds of single-modal networks are trained separately and the system recognition accuracy is improved through model integration,which avoids the interference of complex background environments to a certain extent.