| Sign language recognition is the process that using computer technology to translate sign language action, it is an important field in human-computer interaction research. The situation of communication barriers exist between deaf and normal person and the rising demand for natural and convenient human-computer interaction make researching and applying sign language recognition technology to real world more and more important. Sign language sentences which are used in deaf’s communications are made up of isolated sign words, thus the study of isolated sign language recognition is the basis and prerequisite for constructing a continuous sign language recognition system and has an important value.The Kinect device can output color and depth images and capture all the movements of body joints. When compared to traditional data acquisition devices, Kinect has more advantages in sign language recognition area. In this paper we use Kinect as an input device and investigate the isolated sign language recognition based on it, then we build an isolated sign language recognition system. The main contents of this paper are:1. Extracting the discriminative sign features. By analyzing the original Kinect skeletal tracking information and color information, we design sign features which contain hand trajectory feature and hand-shape features. Hand trajectory feature is the normalized points of original trajectory, which include speed, scale and position normalization. Hand-shape features include grayscale, SURF (Speed-Up Robust Features) and HOG (Histogram Of Gradients) feature.2. The isolated sign language recognition algorithm. This paper introduces the distinctive dictionary learning and sparse representation based recognition algorithm and apply it into the recognition of isolated sign language. We proposed an improved classification algorithm based on joint dictionary which utilize the distinctive information contained in the learned joint dictionary. The proposed classification algorithm can improve classification accuracy.3. An isolated sign language recognition system. We design and implement a real-time and effective system for isolated sign language recognition, the proposed system obtains an average recognition accuracy of98.61%on the collected sign database which contains72Chinese isolated sign words. We design several experiments for validating the effectiveness of our sign features and the proposed recognition algorithm. Results show that when combining trajectory and HOG feature as sign features, system can obtain the best classification result. And when investigating classification accuracy, the improved classification algorithm is superior to original classification algorithm. The comparing results with DTW (Dynamic Time Warping) and HMM (Hidden Markov Models) show that the proposed system is suitable for isolated sign language recognition task. |
