Research On Lip Segmentation Based On Image Sequences

Using visual information in automatic speech recognition has aroused the interest of many researchers in recent years because the visual information will help enhancing the robustness of the system.Lip segmentation is the important step in lip reading system.This thesis presents the studies of lip segmentation and new algorithm for this issue is presented in this thesis.Since human speech is continuous in natural,lip sequences rather than isolated lip images need to be processed.However,most of the current lip segmentation algorithms utilize isolated images and neglect the underlying connections between consecutive frames.In this paper,a fuzzy clustering based algorithm called moving-information-guided fuzzy C-means algorithm is proposed to exploit the above connection and to extract the lip region from lip image sequences.The proposed algorithm is based on the following facts.First,lip's movement is continuous in nature.Hence,for those pixels in static regions or regions with very slow movements,their membership for lip and non-lip class will be consistent.Accordingly,for those pixels in regions with rapid movements,their membership will not have the above consistency.On the other hand,lip color of the same people does not change much.Hence,a penalization term is used in the objective function to penalize the variation of lip centroid during his/her lip sequences.In the proposed algorithm,we adopted optical vector flow calculated by the Lucas-Kanade algorithm to describe the moving information.Then such moving information and the reference lip color information are seamlessly incorporated into the proposed FCM framework to improve the stability of the lip segmentation algorithm.Based on the experiments on a lip sequence data set,it is demonstrated that the proposed method is more suitable to process lip sequences and achieves a high level of accuracy compared with some state-of-the-art approaches.