Research On 3D Facial Landmark Detection Method Based On Deep Learning

With the great development of deep learning in the field of computer vision,various face image processing tasks have been widely used in real life.Among them,facial landmark detection plays a key role in face recognition,expression recognition,face reconstruction and so on.Facial landmark detection,also known as facial landmark locating or face alignment,refers to locating the key area of the face,including eyebrows,eyes,nose,mouth,facial contours,etc.for a given face image.Great achievements have been make in facial landmark detection for the last decade,especially in 2D facial landmark detection.However,some difficulties lie in certain complex scenarios,such as large pose or occlusions.In order to solve this limitation,3D facial landmark,which offers more expressive and occlusive information than its 2D counterpart,is gradually studied by more and more researchers in recent years.However,3D facial landmark detection still face great challenges in processing efficiency,model size and algorithm implementation due to an increase in dimensionality for 3D space.In view of the above problems,this paper has fully studied the existing 2D and 3D landmark detection methods,and proposes a series of improved methods and verified the feasibility of the improvement through experiments.The main research contents and contributions are summarized as follows:(1)We gain the landmark 3D location from a single 2D image directly.(2)We encode the 2D coordinates of landmarks in heatmap representation,which records the likelihoods of landmarks location.Compared with the direct regression of coordinates,the heatmap representation is more conducive for model learning.Moreover,instead of encoding a landmark to a heatmap in traditional heatmap methods,a compact representation is adopt to encode the whole 2D landmark of a face image into a single heatmap,which fix the number of heatmaps and reduce the parameters and accelerate the process.(3)We design a two-stage encode-decode network,the first network encodes the 3D landmark location in comebined heatmap,then the second network regresses the coordinates,which decodes the comebined heatmap for the landmark 3D coordinate vector.Formally,the whole network is trained in an end-to-end manner through cascading the two subnetwork.(4)We extend the 2D landmark heatmap representation to 3D,innovatively present the combined heatmap of three channels,and each 2D image corresponds to a combined heatmap.The representaion reduces the space and complexity required for coding while maintains the connection between the landmark 3D location,resulting in easier learn and process,higher accuracy and speed and smaller size of the model.