Research On Several Issues About Face Keypoints Detection

Facial feature point detection,as a fundamental part of facial analysis,is a classical and crucial problem in the field of computer vision.Based on accurate feature points,applications such as face pose estimation,expression analysis,and beauty effects can be realized.The performance of the detection algorithm relies heavily on the annotation of the feature points,and accurate annotation is the basis for training feature point detectors.Therefore,it is of great significance to improve the traditional annotation method to make it more flexible and efficient.Furthermore,after years of development,the detection scene of facial feature points has evolved from a simple controlled environment to a complex uncontrolled environment,which brings a series of challenges,such as gesture expression changes,illumination changes,occlusion and low resolution,etc.These have become a huge obstacle to the performance improvement of the facial feature point algorithm.This thesis has carried out researches mainly on the annotation,detection and application of facial feature point,and solved several problems in the existing methods through a series of improved algorithms.The main research contents of this paper are as follows:1.A multi-mode annotation platform of facial feature point based on 3D assisted model is developed to realize fast and accurate annotation of facial feature points.The platform realizes the three-dimensional variable model fitting of the face image,and completes rough annotation of the feature point based on the three-dimensional auxiliary information supervision.In addition,the image edge is used as the target template point to register the current feature points so that the accurate annotation is realized.The platform provides different annotation modes to help the labeling staff to achieve efficient and efficient labeling and build custom data sets.2.The advantages and disadvantages of two regression methods in the facial feature point detection algorithm based on convolutional neural network are analyzed,and two new methods are proposed.One is based on numerical coordinate regression and the other is based on heat map regression.The residual network model based on improved residual unit and the serialized prediction model based on intermediate supervision are adopted,and loss function of both architectures is improved.Furthermore,the pose-based data balancing algorithm is implemented by resampling the training samples,which solves the problem of uneven distribution of faces in the dataset to some extent.The connection between two regression methods is established through the soft-argmax function,so that the improved heatmapbased regression method can achieve end-to-end training while taking into account performance.Compared with the traditional heatmap-based,the regression method eliminates some of the data preprocessing process,which in turn reduces training time.3.The application research of face image synthesis based on face keypoints detection is carried out.Depending on whether the 3D face model is used,two virtual face-changing methods are designed,and the experimental results are compared and analyzed.The human face and the facial features "mask" are generated based on the locations of the feature point,and the face skin color adjustment and makeup synthesis application are completed.