The Research Of Real-Time Dynamic Sign Language Recognition Based On Kinect

Gesture recognition is one of the hot issues in the field of computer vision. Sign language as a special gesture is the "language" used in the communication between the deaf, the deaf and normal people. Automatic sign language recognition can not only create an exchange platform for the deaf and normal people, but can also improve the sensing ability of the computer and increase human-computer interaction. This paper achieves the goal of recognizing dynamic sign language by using Kinect, a somatosensory camera which can return depth information, along with the identification of the human skeleton position.Dynamic sign language can be described by trajectory curve and key-hand manner, but in fact most of the sign language can be identified by matching the curve of the track. Therefore, a grading matching strategy to recognize dynamic sign language is presented in this paper. Level one is completed by measuring the similarity of the trajectory curve; if the results can be identified by level one, then the identification process will be completed, otherwise enter level two, which is to recognize the key hand type of the sign language. To achieve that goal, firstly Kalman filter which can correct the hand position detected by Kinect is introduced in this paper, while a precise segmentation algorithm which combining depth information returned from Kinect and Kalman filter is proposed too; and then a curve normalization algorithm to eliminate the different movements caused by time and people is also presented, and a key frame detection algorithm based on trajectory curve density, and extracts feature hand-type from key frame to complete characteristic description of sign language also proposed in this paper; finally, DTW algorithm has been improved by combining key frame information in level one matching to make it more suitable for gesture trajectory curve similarity measure, critical hang shape is recognized by using template matching methods in level two matching.Based on the above algorithms, using C# as the development tool, this paper designed a real-time dynamic sign language recognition system, which can realize the recognition of 60 common dynamic sign languages, and can be used for non-specific groups.