Research On Video Lip Visual Features Extraction And Tracking Technology

With the rapid development of science and technology and artificial intelligence, the human-computer interaction technology has become a multi-media and multi-mode technology that from computer-centric to people-centric, and more and more researchers focus on lip reading technology. Lip visual features extraction and tracking technology, as the important part of lip reading technology, has high research and application value. So this paper mainly includes two aspects, lip segmentation and visual features extraction technology and tracking technology. These two aspects are introduced respectively in the paper.For lip segmentation and visual features extraction technology, face can be detected by OpenCV, and then lip region can be extracted from the face image. A algorithm for lip contour extraction that combines the merits of the pixel-based model and the parametric model is presented. The mouth key points are extracted by lip corner detection, the improved Jumping Snake algorithm and four different gradients to deal with the lip contour. And then the accurate and complete lip contour can be acquired using three different curves based on the key points. The geometric visual features is also extraction, so we achieve lip segmentation and visual features extraction completely.For lip tracking technology, the lip key points are tracked by Mean Shift algorithm. However, lip key points can be tacked unsuccessfully sometimes because of the influence of the external environment, so self-adapting scale update is added to the original Mean Shift algorithm to improve the key points tracking and avoid the failure condition to the greatest extent. Experiments which are carried out in this paper show that the improved algorithm can obtain satisfied results and prove that the accuracy and efficiency of this algorithm.