Research On Facial Landmark Detection Based On Heatmap Regression

Facial landmark detection is an important topic in the field of computer vision.Accurate facial landmark detection is of great significance on many face analysis tasks,for example,head pose estimation,face synthesis and expression recognition.In the last decades,with the development of deep learning,facial landmark detection has come a long way.However,under unconstrained circumstance,the high variability of poses and possible occlusions makes it a particulary challenging task even today.At present,facial landmark detection method based on deep learning can be further subdivided into two categories: one is based on coordinate regression,which is directly mapped from image to numerical coordinate by neural network,and the other is based on heatmap regression,which realizes the mapping from image to heatmaps,and then infers numerical coordinates from heatmaps.This paper makes an in-depth study of the facial landmark detection based on heatmap regression,and mainly puts forward:(1)A facial landmark detection method based on improved hourglass network and sample balance.An improved stacking hourglass network structure is used,and the adoption of deformable convolution makes the network have adaptive receptive field.Also,a pixel-by-pixel focus loss function combined with sample balance was proposed.Different weight were put on different samples according to the Euler angles of the head,which makes the method pose robust.(2)A facial landmark detection method based on lightweight network and shape constraints.A lightweight network structure is proposed,which combines with attention module and coarse-to-fine intermediate supervision.A data post-processing module based on shape constraints is designed to further correct the detection results of facial landmarks,which divides detected landmarks into reliable and unreliable according to the output probability of heatmaps,and then uses the simplified shape dictionary to re-infer unreliable landmarks with reliable ones,thus makes the method occlusion robust.Through experimental results on public databases,the proposed methods achieved good performances both under the constrained circumstance and the wild circumstance,in comparison to the state-of-the-art methods.