Facial Landmark Positioning And Tracking Based On Spatial-temporal Deformable Networks

Human is the main content in images and videos,and was taken as the central focus of media content analysis by emerging visual technologies.Facial landmarks are essential for understanding and analyzing human facial behaviors.Meanwhile,the technology of facial landmark positioning is the basis and premise of facial analysis technologies like face recognition,expression analysis and head pose estimation.In recent years,facial landmark positioning methods based on deep learning have performed well under constraint conditions,but are still limited by the stable spatial structures of conventional convolutional network,as well as the stable temporal structure of regular time-series network.Existing methods are difficult to capture the most significant information for landmark positioning.In response to the problem,the main work of this article are brought up as follows:1.To deal with relevance of the spatial structure of the faces in static images,a spatial deformable network is proposed.The method aims to learn a deformable convolutional module,which fully considers the facial deformation caused by head posture,different expressions and occlusions,and can explicitly mine spatial geometric information for various facial shapes,further extracting the most effective feature information for facial landmark positioning task.Experiments show that this method can effectively extract the structured information of the face,and the NRMSE on the widely-used benchmark 300-W is reduced by about 10%.2.To deal with high-order time-series modeling of faces in videos,a temporal deformable network is proposed.The method aims to make the model's encoding of the current frame include historical features of the past few frames via the temporal deformable module,and by establishing the inter-frame pairing relationship,the model can adaptively flow across and choose the more important past frames for the landmark tracking in current frame.The experimental results show that the method can accurately describe the high-order timing relationship,and the NRMSE on the widely-used benchmark 300-VW is reduced by 8.1%.3.According to the proposed spatial-temporal deformation network,based on the principles of usability and reliability,a facial landmark positioning and tracking program is set up,whose main functions contains training process display,parameter adjustment,and result visualization.The program can be used to verify the validity and practicality of the model proposed in this paper.