Research Of Gesture Recognition Based On Kinect

The study of sign language recognition technology is popular.Vision-based gesture recognition technology is currently a hot topic in the field of HMI.The traditional sign language recognition method requires seriously of environment background,image acquisition equipments and gestures.Therefore,this thesis use the Kinect device to collect gestural images,aiming to reduce the influence of complex background and illumination conditions,and research the gesture recognition algorithm based on the collected images.The main research process of this thesis is consist of four stages: the establishment of gesture image database,gesture image preprocessing,gesture feature extraction and gesture recognition.First,the gesture image database is set up by the Kinect.Several hand gesture images of different illumination conditions,different backgrounds and different angles are collected,which is prepared for the following operations.Second,the collection of gesture images is preprocessed,including the following four parts: gesture Region of Interest(ROI)acquisition,image filtering,morphological processing of gesture images and gesture image contour extraction.The hand ROI area acquisition is according to the characteristics of the Kinect,the color region is obtained after the color detection of the color image,and the foreground image which is segmented from the depth histogram threshold is carried out to obtain the ROI area of the hand;The image filtering part using median filtering remove the interference of noise in the image;The morphological process of gestures uses first open and then closed the morphology;The gesture image contour uses use the Canny operator edge detection method to extract the contour of hand gesture.Furthermore,Hu moments,roundness,rectangularity and length-width ratio are used to extract the gesture features.The fusion feature data is used as the input of the classifier.Finally,three classification methods i.e.Support Vector Machines,Neural Networks and Random Forests are used to identify and analyze the extracted gesture features and analyze the results of the recognition.The experimental results show that using the method to gesture recognition in this thesis,eight kinds of static sign language are identified.The average recognition rate of Support Vector Machines is 97.4%.Therefore,the Support Vector Machine is used in the experiments of different illumination conditions,different deflection angles and different rotation angles have been carried out to verify the robustness of the gesture recognition algorithm introduced in this thesis.