Monocular Gesture Recognition System Design For Human-computer Interaction

In our daily life,we usually use the keyboards,touch screens,and mice to interact with our computer.In order to take advantage of these tools,we have to accept some inhumane trainings to adapt to these devices.In recent decades,there is a new way of interaction named hand gesture recognition that can help us easily input information in a non-contact,more natural,more humane,and simpler way.As a result,they are widely applied in smart homes,internet of things,smart office,hospital security,and smart education.So,how to use gesture recognition for interaction is still a worthwhile research topic.This article mainly completes the work of vision-based static and dynamic gesture recognition.The static gesture recognition consists of three parts,including hand shape segmentation,hand shape feature extraction,and hand shape classification.In terms of gesture segmentation,in order to reduce the interference of the complex light and background on hand-shaped segmentation and improve the segmentation precision of hand-shaped objects,we use the improved SSD(Single Shot MultiBox Detector)neural network model to extract the static hand shape instead of using the traditional segmentation algorithm such as frame difference method,skin color model method,template matching method,etc.The SSD network model based on the two-dimensional convolutional neural networks has the multi-level convolution kernels and target pre-selection boxes property.Since the scale of multiple convolutional output layers which have different sizes of feature map,so it can be used to detect the targets of different sizes.Besides,using the target preselected box can ensures that the output target can be regressed to the position of the target frame and the target class.Experiments show that the algorithm can greatly improve the accuracy of static gesture segmentation.Through the experimental analysis in the complex scenario,the accuracy of hand gesture segmentation using the SSD network model is improved by 20% and 15% respectively compared with the traditional frame difference algorithm and skin color model algorithm.In the terms of hand features extraction,we can extract the HOG(Histogram of Oriented Gradient)features.For hand classification,we use a hybrid classifier consisting of a SVM(Support Vector Machine)and Softmax to classify and recognize hand-shaped HOG features.Experiments show that the system can accurately recognize these different hand shapes(static gestures).For the 10 custom gestures including the digits of 0 to 9,the average recognition accuracy can reach 89.6%,which is higher than the effects of the traditional algorithms such as frame difference method and skin color model method.In addition,for the dynamic hand gesture recognition,we use our network model consists of LSTM(Long Short-Term Memory)network and 3D-CNN(Three Dimension Convolution Neural Network)to process dynamic gesture to learn the temporal and spatial characteristics of multi-frame images,and then use the Softmax classifier to classify the dynamic hand gesture feature.The experimental results show that the system's gesture recognition stability and precision rate are quite good.Our dynamic gesture recognition algorithm supports a total of six kinds of dynamic gestures including upward gesture,down gesture,left gesture,right gesture,shocking hands gesture,vertical thumb gesture and the average recognition rate of all the gestures can reach 85.8%,significantly higher than the result of traditional TLD(Tracking Learning Detection),HMM(Hidden Markov Model)and other algorithms.Although the average processing time of each dynamic gesture is slightly inferior to the conventional algorithm,it does not affect the actual user experience.