The Recognition Of Hand Gesture Motion Behavior Based On Deep Learning

With the more and more applications of robots,human-computer interaction has always been a hot field.Human and machine usually communicate through voice,gesture,EEG tracking and visual sensing,among which hand gesture can convey abundant information to the machine.However,the diversity and complexity of hand gestures,as well as the differences in time and space in the process of motion,make it difficult to recognize.Traditional recognition methods rely on wearable devices and a variety of sensors to locate and identify gestures.These methods have many disadvantages,such as high cost,large external impact and low accuracy.The recognition method based on deep learning can greatly over these difficulties.This paper studies the algorithm of convolution neural network for gesture classification and recognition,and completes dynamic gesture recognition with two-stream convolution neural network.The specific contents are as follows:Firstly,one convolutional neural network model is built based on LeNet-5 network.The training of the neural network model is completed by using the self-captured gesture data set.By designing comparative experiments,network layer number,input size and convolution core size are determined.Based on the recognition accuracy and training time,the best network model is built to accomplish the task of static gesture recognition,and two error processing schemes are designed to improve the recognition accuracy of the network.Then,we used optical flow method to extract the dynamic features of motion gestures in video,and trained another convolution neural network model by using optical flow graph containing the time dimension information of gesture motion.Two convolution neural networks are connected in parallel to form a two-stream convolution neural network which can extract the information of time and space dimension in gesture motion video to recognize gesture motion behavior.The results show that the two-stream convolution neural network works better than a single convolution neural network on gesture recognition.Finally,this paper designs and compiles a gesture recognition software and applied the trained convolutional neural network model to the recognition system.The camera is used to collect data,and the data and results are displayed and manipulated with the image interface.The gesture movement behavior recognition is completed in the form of visualization.