Research On Signer Adaptation In Chinese Sign Language Recognition

With the invention of the computer, pattern recognition appeared and blossomed.As a hot research area in pattern recognition, sign language recognition has been paidmuch attention by many researchers. Sign language recognition is to automatically tran-scribe sign language to texts or speech by computers. It is of great value both for realapplication in society and for theoretical research. First, sign language recognition canbuild a bridge between the hearing impaired and the hearing society, which promotes thesociety's development harmoniously. Second, sign language is a more structured type ofsigns. Compared to other types of sign analysis, recognition of sign language is relativelysimple. Sign language recognition can serve as a test bed for more general research onsign analysis. Moreover, sign language recognition is related to the areas of computervision, pattern recognition, machine learning and intelligent human-computer interactionetc. Sign language recognition can help to solve the similar problems in these areas.After many years of research, signer dependent sign language recognition systemshave achieved good performance. However, the performance decreases drastically whenthe test signer is unregistered in the training data set. Collecting enough data from dif-ferent signers to train signer independent models can solve this problem to some extent.Nevertheless, the models are difficult to converge because data from different signersare extraordinarily diverse. Moreover, the distributions of signer independent model pa-rameters are plain, and the acceptable performance can be achieved on lots of signers.However, the performance of signer independent models is not as perfect as that of signerdependent models for a specific test signer. Adaptive sign language recognition utilizesdata from a new signer to tailor the initial models, and the tailored models can bettermodel the new signer. The method accords to the mechanism that people perceive theworld from generality to specialization.This dissertation aims to solve the problem of signer adaptation. According to theavailability of the adaptation data's labels, signer adaptation can be classified into super-vised signer adaptation and unsupervised signer adaptation. For supervised signer adapta-tion, the labels of the adaptation data are needed, so the data collecting process is explicit.Explicit data collecting needs the intervention of the user, which is not acceptable to some users. Therefore the core problem in supervised signer adaptation is to modify the param-eters using as less data as possible. For unsupervised signer adaptation, the labels of theadaptation data are needless, so the data can be collected implicitly at the same time whenthe user is manipulating the system. However, the data must be labeled before they areused for adaptation. Therefore the core problem in unsupervised signer adaptation is tomodify the parameters with the huge amounts of unlabeled data effectively.For supervised signer adaptation, the adaptive sign language recognition methodsbased on basic units extraction and that based on exemplar extraction and MAP/IVFSare proposed. Inspired by that sign language recognition based on etyma can achievecomparable results with that based on words, we propose signer adaptation based onetyma. Experimental results showed that signer adaptation based on etyma could bothpreserve the recognition rate and save the adaptation data greatly. Further more, there aresimilar segments in different Chinese sign language (CSL) words. By clustering meanvectors into clusters the CSL words can be coded. Using representative words'samples,the virtual samples of the whole vocabulary can be generated. Using these data the modelscan be adapted and the adapted models can achieve higher performance. Experimentalresults showed that the amount of adaptation data can be decreased further. To reduce theamount of adaptation data needed further, signer adaptation based on exemplar extractionand MAP/IVFS is proposed. A mean vector subset can be selected by clustering, and thecorresponding word subset can be directly formed. The word subset can represent the newsigner's signing characteristics. Using the samples in the word subset, the correspondingmodels can be adapted. The other models can be estimated using the adapted models andthe correlation among the models.Though the supervised signer adaptation can adapt the models with small amountof data, the explicit data collecting process is indispensable. However, unsupervisedsigner adaptation can collect the adaptation data implicitly. For unsupervised signeradaptation, the combination of simplified polynomial segment model (SPSM) and hiddenMarkov model (HMM) for unsupervised adaptation and unsupervised adaptation basedon hypothesis-comparison guided cross-validation (HC-CV) are proposed. HMMs aresuitable to model the words which have obvious state transitions, and are not suitable tomodel those which have not obvious state transitions and change frame by frame. This isbecause that in HMMs the observations in the same state are supposed to be independent and identical distributed. SPSMs are suitable for modeling the other type of words inthat SPSMs can model the correlation between frames. Combining SPSMs and HMMsto label the unlabeled data can decrease the noise rate of the adaptation data set, whichleads to the improvement of the adaptation. In conventional self-teaching adaptation, themodel set that is used to label the unlabeled data and the model set that is to be adaptedare same, which leads to the error reinforcement and the over fitting. By introducingcross-validation to unsupervised adaptation, the problems encountered can be relieved.Applying hypothesis comparison for unsupervised adaptation, the labeling right rate canbe improved. By this way the adaptation can be more effective.By solving the signer adaptation problem in CSL recognition, the preparation forthe application of the CSL recognition system in daily life has been supplied. Moreover,signer adaptation in CSL recognition can help to solve the adaptation problems in otherresearch areas.