Human Mouth-state Recognition Based On Sparse Representation

A human auditory system can easily distinguish someone’s words and detect thenumber of audio sources from acoustic mixtures generated in a complex environmentbut an intelligent robot’s cannot. In order to help the intelligent robot auditory systemseparate the speech signals and determine the number of audio sources, the visualinformation, i.e. human mouth-type image, is used. A sparse representation basedclassification algorithm is proposed to recognize the human mouth-state.The human mouth-state recognition is studied in two aspects, discriminativedictionary learning and sample optimization. On the one hand, the label consistentK-SVD (LC-KSVD) algorithm is used to simultaneously learn a single discriminativeover-complete dictionary and an optimal linear classifier. Besides the reconstructionerror penalty term measuring the representational power, both discriminativesparse-code error and classification error are added into objective function to increasethe discriminative power. This overcomes the disadvantages of K-SVD which focusonly on the representational power and learn the dictionary and classifier separately.On the other hand, A sixteen-point lip model with some geometric constrains isdefined to describe the lip contour. The mouth region is extracted and warped intothis standard lip model by an image warping algorithm and the pixels outside themouth region are wiped off to geometrically normalize the human mouth-state sample.The dictionary will be generated by stacking the optimized training samples orlearned by LC-KSVD or K-SVD algorithm. Among the two aspects’ research, twohuman mouth-state recognition algorithms are proposed. One is the humanmouth-state recognition based on learned discriminative dictionary and sparserepresentation with homotopy. The other is the human mouth-state recognition basedon image warping and sparse representation with homotopy. Both of them use thehomotopy algorithm in the sparse coding stage to overcome the non-convergencedisadvantage of greedy algorithms like OMP due to the over-completeness of thedictionary. Experiments are carried out with the relevant human mouth-state imagesdownloaded from Google online. Compared with several state-of-the-art methods, theproposed methods in this dissertation are more efficient and effective.