| Human Computer Interaction is one of the key elements of Multi-Media. In this thesis, we mainly study the recognition of American Sign Language (ASL).; The first step is hand segmentation. Compared with snake, level set can naturally deal with objects with significant protrusions and topological changes. However, either the original level set algorithm is heavy computational and far from a real-time application, or the existing fast marching format is much lost in performance. In this work, a novel fast and robust level set algorithm is proposed. It keeps all the advantages of the classical level set, and has a much faster convergent time.; Hand gestures are intended hand movements, which involves both the static hand postures and dynamical hand movements.; For hand posture recognition, we propose two new schemes: 3D hand model-based and appearance-based. In the model-based approach, a 3D model is built for each hand spelling. Then a factored sampling process is used to approximate the posterior probability for posture verification. This is the first report in the literature of using a model-based approach for ASL recognition in a large vocabulary. In the appearance-based approach, we present a novel PCA (Principal Component Analysis)/MDA (Multiple Discriminant Analysis) scheme. Based on the Expectation-Maximization (EM) algorithm, we introduce three methods to estimate the parameters for a crude classification in the PCA layer during training.; Next, Hidden Markov Models (HMMs) are described for hand movement recognition. Normally, one sign is modelled by one HMM. However, in this thesis, 9 movement phonemes are modelled by HMMs instead of hundreds in word modelling. A three-channel PaHMM system is introduced to model the changes of the right and left hand positions and the changes of the postures of the right hand. The outputs of these three channels are mutually weighted between a pair of ASL signs, such that a more discriminative channel plays a more important role for recognition. Finally, based on the HMM framework, our early work for gesture spotting is introduced and a view invariant recognition scheme is presented.; Real experiments on a 110-sign vocabulary are tested. The proposed methods show their real potential for application on a large vocabulary. |
