HMM Based Temporal Sequence Clustering And Recognition

In the first part of this paper, a gesture recognition system is designed and implemented. The gestures are pre-defined for browsing 3D objects. After the low level image processing and feature extraction, HMMs are used to describe all kinds of gestures. Then a Threshold model is introduced to automatically divide a continuous image sequence into meaningful gestures and atypical gestures. The system can recognize the type of a typical sequence and reject atypical ones.During the researching, some weak points were founded in the above gesture recognition system, and two amendments are made. A pervasive applicable feature extracting method is presented. The method is based on the distribution of points, and PCA is used to reduce the dimension of parameters according to the training data set. The key achievement of my research is summarizing the gesture as a temporal sequence, and unsupervisely clustering, model building and recognizing methods are designed to analyze it. Based on entropy minimization, a novel HMM based hierarchical clustering method was invented, aiming at the grouping of temporal sequences. After the traditional Baum-Welch training step, TOM (Transition Occurring Matrix) is calculated and its modified version WTOM (Transition Weighted Matrix) is used as the only feature of the sequence in clustering. Thus the original clustering problem is converted to a relatively easy high dimensional data clustering problem, which is solved by NCut method. Using of WTOM to compute the similarities between sequences is both time-efficient and robust in comparison with k-mean alike methods. After clustering, the structure of the dataset is derived, and the model performance is enhanced as well. The method has been tested on unsupervisely learning of hand gestures, event analysis in video surveillance, and model learning of handwriting. Good results are achieved in all applications.