Research On Dynamic Lip Segmentation And Tracking Technology

With the development of science and technology, lip segmentation and tracking, a sub-stepof lip reading technology, becomes more and more important in the fields of pattern recognitionand human-computer interaction. In order to develop an accurate and stable algorithm for lipsegmentation and tracking technology, we separate the realization of this technology into twosteps, lip segmentation technology and lip tracking technology. Thus, in this paper, we talk aboutthese two steps respectively in different chapters.For lip segmentation technology, the multi-class, shape-guided FCM (MS-FCM) clusteringalgorithm is first introduced and studied in the paper. For this algorithm, several clusters are setfor different part around the lip region which facilitates the separation of lip and backgroundpixels. And a spatial penalty term considering the spatial location information is introduced andincorporated into the objective function such that pixels having similar color but located indifferent regions can be differentiated. Experimental results based on AR face database show thatthis algorithm provides accurate lip segmentation. Secondly, a novel algorithm for lip contourextraction that combines the merits of the pixel-based model and the parametric model ispresented. Lip corner detection is the first step of this algorithm, utilizing a three-curve lip modelto describe lip contour. The "Improved Jumping Snake" algorithm is used to extract feature pointsfor lip model implementation, which makes the novel algorithm more accurate and flexible. It isalso made more robust with the introduction of geometric constraints.Experiments also show thatthis approach with geometric constraints provides satisfactory results. These two algorithmsabove are realized by absolutely different ideas, but they both can also serve as an initial attemptfor further applications.For lip tracking technology, we presented a systematic lip tracking algorithm based on theMS-FCM algorithm, feature points extraction and contour estimation which are key ideas in thepart of segmentation. We first make the MS-FCM algorithm adapted to the lip sequencesegmentation by using the final membership matrix of the previous lip image to predict andinitialize the membership matrix of the current one. Then we do training to obtain a mean shapesequence of the lip sequence and its main deviations by manual labeling the training set. Aftertraining, we try to extract feature points from the original lip segmentation sequence, and alignthese feature points with the obtained mean shape sequence and relocate those unreasonablepoints. Finally, the contour of tracking result is estimated using least-square estimator. Similarly,experiments which are carried out in this paper have proved the accuracy and efficiency of thisalgorithm.