The Research Of Real-Time Gesture Recognition System For Non-specific Population

Sign language is a way of communication between the deaf and normal people.As the vast majority of normal people know little about sign language,it is difficult for them to communicate with the deaf.Realizing real-time sign language recognition can help deaf-mutes build normal social relationships with other members of society.This paper select the Kinect for Windows camera as input device and use the depth information and the skeleton position information which are provided by Kinect to realize real-time recognition of dynamic sign language.Dynamic sign language usually regards the shape of the gesture as its meaning.It can be seen as a combination of several practical,static gesture.These static gesture are the key poses of the dynamic sign language.In this paper,an adaptive method of extracting key poses is presented;and then the skeleton position information and the features of the hand are used jointly to describe these key poses;Random Forest is applied to classify the key poses.In the process of classification,the unequal frequency of the different key poses,some of which even vary widely,causes the unbalance between the sample set of the key poses.In this paper,KM-SMOTE algorithm has been improved to resolve this problem;finally,using the characteristic information of the key poses,the Hidden Markov Model is used to accomplish sign language recognition with the sequences of the key poses.Based on the above algorithm,this paper designed a real-time dynamic sign language recognition system,which can realize the recognition of 20 dynamic sign languages.The system is composed of the foreground sign language recognition exhibit and background sign language video management.Its recognition rate is 90% for person who participated in the sample collection,and is 85% who don't participate in the sample collection.