Joint 3D Head Pose And Face Landmarks Regression Based On Classification-guided Method

The development of internet technology opens a digital era,extracting the high-level knowledge from a large amounts of data with machine learning method and depth learning method has become a major research focus on human-computer interaction.The key of human-computer interaction is to identify the specific parts of the human body according to the different interaction requirements.As a convenient and friendly identity feature,biometric technique emerges as the times require.The existing mature biometric technique includes iris recognition,fingerprint recognition,speech recogni-tion,gait recognition,face recognition,and so on.Since human face is one special biometric feature because of its convenience in acquisition and no-infringing,the study of human face can be easily accepted by the subjects,therefore,the research in this field is becoming more and more mature.Estimation of human head pose and detection of face landmarks such as eye corner-s,nose tip,mouth and chin are of central importance to face analysis.These two prob-lems have achieved significant progress on images.However,image-based methods always suffer from illumination,pose angle and occlusion,which lead to many limita-tions.In recent years,with the low-cost and acceptable precision of RGBD technology like Microsoft Kinect and Intel Realsense,head pose estimation and face landmarks detection methods based on depth data attract more and more attentions due to the rich geometry information of the depth images.Head pose estimation and face landmarks localization have been separately studied for many years,but the result of head pose estimation can provide a good global spatial transformation for face landmarks local-ization,while the structure of face landmarks can reflect the head pose vector,so how to combine the two optimization is one core problem in this paper.In this paper,we propose a Joint 3D head pose and face landmarks regression based on classification-guided method.First of all,classification-guided method is to divide the head pose space into several classes,and trains the specific face landmarks local-ization model for each type.In this way,we can guarantee that the head point cloud is relatively consistent in the same pose space,which is helpful to improve the stability of face landmarks localization algorithm.Secondly,we present a joint optimization con-cept in cascaded random forest framework,the pose regression result with an template can provide supervised initialization for cascaded face landmarks regression,while the estimated face landmarks can also help to refine the head pose in each stage.Finally,we give a 3D face database with well labelled head pose and face landmarks,which contains the different identity persons,each with different expressions in various head poses.Plenty of experimental results demonstrate the effectiveness and efficiency of the proposed method.The method presented in this paper is more accurate than the exist-ing methods in two commonly used 3D databases which called as BIWI database and B3D(AC)2 database.In addition,our method has the generalization capability in other application about pose estimation and location of key points.