Lip Region Segmentation And Modeling In Complex Environment

Recently,with the rapid development of machine learning and pattern recognition,more and more attention has been paid to human-computer interaction and authentication technology based on human biological characteristics.Visual speech recognition authentication is one of the key technologies.Lip image segmentation technology can provide a lot of visual information for visual speech recognition.It is the most basic technology in visual speech recognition authentication technology.Lip image segmentation module is a sub-module of many applications,such as lipbased authentication system and lip reading system.Accurate lip segmentation has great application value in education,security and other fields.In this paper,a lip segmentation method based on deep neural network is proposed to achieve accurate lip segmentation in complex environment.In view of the diversity of the environment,it is necessary to add annotation information to ensure the accuracy of segmentation results,while manual annotation is time-consuming and laborious.To solve this problem,this paper proposes a low-cost annotation method,which combines face key point detection algorithm and maximum stable extremum region algorithm to label the inner and outer lip contours.However,both manual annotation and low-cost annotation proposed in this paper will introduce different levels of annotation noise,which will affect the accuracy of segmentation results.In consideration of the annotation noise,this paper proposes a lip region segmentation network based on full convolution neural network,which integrates multi-scale information.This method consists of two relatively independent neural networks,one is used to obtain a large lip region,and the other is used to reduce the segmentation error caused by annotation noise.At the same time,a new loss function is proposed,which makes full use of the relationship between consecutive frames,so that it has stronger robustness to annotation noise and unbalanced data.The experimental results show that compared with the existing lip segmentation methods,this method can achieve better performance.