Research On Pose And Occlusion Robust Facial Landmark Detection Algorithm

With the advent of artificial intelligence era,face-oriented computer vision has developed rapidly.Facial landmark detection,as a basic technology for high-level visual tasks such as face recognition and expression recognition,is a research hotspot in multimedia community.Since the rigid or non-rigid deformation of face appearance,traditional facial landmark detection algorithms are usually sensitive to some challenging scenarios,such as large-angle pose,and partial occlusion.Recently,the development of deep learning has brought dawn to facial landmark detection.Therefore,this dissertation focuses on deep convolutional neural networks,supplemented by traditional algorithms,to further study the problem of facial landmark detection.The main contents of this work are as follows:(1)Facial landmark detection based on face region normalization and deformable hourglass network.Hourglass network is a classic network for facial landmark detection,however it is sensitive to large-angle pose variations.To improve the robustness of hourglass network,a twostage model is proposed for facial landmark detection.First,by using a spatial transformer network to warp face image to a canonical pose,the conventional procedure of normalizing the face region by performing Procrustes analysis based on the detected landmarks and the mean shape is simplified.This is more advantageous for subsequent face alignment.Second,motivated by recent proposed deformable convolutional network,deformable convolutions are used in conjunction with hourglass networks,resulting in deformable hourglass network.Compared with original hourglass network,deformable hourglass network achieves large performance improvements while having the almost same amount of parameters and bringing minor additional computation costs.Extensive experimental results based on public datasets 300 W and COFW demonstrate the effectiveness of the proposed method.(2)Multistage model for robust facial landmark detection using deep neural networks.A multi-stage model that is robust to pose and occlusion is proposed which takes advantage of spatial transformer networks,fully convolution neural network and exemplar-based face shape dictionary.First,a spatial transformer network based on the concept of generative adversarial networks is utilized to solve the initialization issues caused by face detectors,such as rotation and scale variations,in order to obtain improved face bounding boxes.Then,a fully convolutional neural network is used for coarsely landmark detection,and at the same time,the positions of landmarks are scored with confidence.Finally,an exemplar-based shape dictionary is designed to determine landmarks with low scores based on those with high scores.Extensive experiments on public datasets show that the proposed multi-stage model can significantly improve the misaligned landmark caused by occlusions or cluttered backgrounds,and outperforms other state-of-the-art methods.