Study Of Stabilization Techniques For Real-time Human Face Landmark Localization

Human face landmark localization is a computer vision task of showing locations of key points,such as specific lineaments of faces and facial organs,given an image.It conveys multi-dimensional image semantic information,e.g.face poses,facial charac-teristics,expressions.This task is very fundamental since it is often used as the prereq-uisite of many complex facial vision task pipelines.As a result,the technique of fast and accurate face landmark localization is of great application value.Recent breakthroughs of deep convolution networks have led the accuracy of face landmark localization on images to saturate to an extraordinary level.The upsurge of video applications also push the requirement of real-time face landmark localization.However,a new problem of landmark jitter emerges when naively applying the tech of landmark alignments for images to landmark tracking tasks for video streams.Land-mark jitter is a phenomenon of undesired fitful movements of certain landmark point while the actual motion of the face,relative to the frames,is along a regular continuous curve.It can really botch user experience of this kind of applications.This study probes the possible causing of the landmark jitter problem,and dis-cusses why improvements of accuracy criteria on current image dataset does not nec-essarily mean better stability on videos.Specialized stabilizing techniques for human face landmark tracking on video streams are urgent.We show how a novel technique called "SMG reverse transformation with adversarial training" helps to ease the land-mark jitter problem.Experiments show that our method can actually tackle the landmark jitter problem without deteriorating accuracy.Besides,we borrow the statistical crite-rion" approximate entropy(ApEn)" from time series analysis to measure the stability of landmarks,which has,in contrary to existing stability measures,no dependency on accurate and stable landmark labels.We qualitatively argue that ApEn matches human feelings about the extent of jittering.