Research And Implementation Of Hand Gesture Recognition Based On Deep Learning

Hand gesture recognition is a topic in computer science to recognize human gestures captured by sensor devices.In this paper,we build a system based on deep learning to recognize hand gestures,static and dynamic hand gestures included.In this paper,research status and significance of hand gesture recognition are introduced at first.We compared different methods and techniques for hand gesture capture and recognition.In second chapter,we use Leap Motion to collect gesture data and try to extract features from these data.The skeleton model of Leap Motion is studied,and we analysis which part of hand can be captured by Leap Motion.We designed the same strategy to collect frames of static and dynamic gestures,serialization and deserialization of frames are also performed in the same manner.Frames of data are extracted into features for training and prediction.Sampling are performed on dynamic gestures to get appropriate sequence length,and we use data augment to handle problem of lack of enough dynamic gestures data.In Chapter 3,we use TensorFlow to build model to recognize static gestures and dynamic gestures,based on feedforward and recurrent neural networks respectively.For Dynamic gestures,we analyzed information morphing and gradients vanishing of vanilla RNN and use LSTM to avoid these problem.And due to different sequence length of each dynamic gesture sample,we expand LSTM in time steps according to their corresponding sequence length so that long term dependencies can be learned by our model.In chapter 4,we studied the visualization system of hand gestures recognition and data collection.We use PyQt as development framework and integrated Leap Motion and TensorFlow into the system.For gesture recognition,it can collect data in real time and predict with pre-trained model.Recognition results can be presented with text or audio.For data collection,the system can control data collection process automatically.And the file system of operation system,on which the leap motion is running,is integrated into our visualization system so that we manage collected data efficiently.