Font Size: a A A

Head Pose Estimation And 3D Face Reconstruction Using Convolutional Neural Networks

Posted on:2022-03-11Degree:MasterType:Thesis
Country:ChinaCandidate:Y Q XuFull Text:PDF
GTID:2518306605471924Subject:Pattern Recognition and Intelligent Systems
Abstract/Summary:PDF Full Text Request
With the development of three-dimensional(3D)point cloud technology,people are no longer satisfied with the two-dimensional(2D)plane image and move towards the 3D field.Depth cameras can capture 3D information from scene and thus are widely applied to human-computer interaction(HCI)and augment reality(AR)which need more reliable 3D information.Since depth cameras become much cheaper and popular with its precision improvement that can be even deployed on smartphones,researchers actively investigate 3D vision using depth information.Compared with depth images captured by depth cameras,called 2.5D information,researchers attempt to transform the depth image into 3D point cloud or capture 3D point cloud data by multiple devices.In this thesis,we utilize the 3D point cloud to directly obtain 3D information from the input data for 3D head pose estimation to avoid the mapping between 2D and 3D.Moreover,we adopt Graph Convolutional Network(GCN)as the network backbone for 3D head pose estimation that performs better than traditional 2D convolution on processing 3D point cloud.Based on them,we propose a 3D face reconstruction network with 3D morphable model(3DMM)and face alignment.We do not use the 3DMM parameters,but utilize 3D head pose and 3D facial landmarks to form a constraint for 3D face reconstruction and alignment.The main contributions of this thesis are described as follows:1.We propose head pose estimation using deep neural networks and 3D point cloud.Unlike existing methods that either take 2D RGB image or 2D depth image as input,we adopt 3D point cloud data generated from depth image to estimate 3D head poses.To further improve robustness and accuracy of head pose estimation,we classify 3D angles of head poses into 36 classes with 5 degree interval and predict the probability of each angle in a class based on multi-layer perceptron(MLP).Although traditional iterative methods for head model construction require high computational and memory costs,the proposed method is lightweight and computationally efficient by utilizing a sampled 3D point cloud as input combined with GCN.Experimental results on Biwi Kinect Head Pose dataset show that the proposed method achieves outstanding performance in head pose estimation and outperforms state-of-the-art ones in terms of accuracy.2.We propose pose guided 3D face reconstruction that uses only 3D facial landmarks and head poses.Most existing 3D Morphable Model(3DMM)based methods predict coefficient parameters of 3DMM under the given single or multiple image input.However,there are only a few dataset available with annotation of 3DMM parameters and the acquisition of accurate 3D model is far from easy.To tackle this problem,we present pose guided face reconstruction to regress an accurate 3D face model without using shape and expression coefficient parameters.We use the 3D facial landmarks and head pose as annotations for a constraint.Different from the 3DMM parameters,3D facial landmarks and head poses can be accurately estimated from the wild images.Experimental results on 300W-LP and AFLW2000-3D datasets show that the proposed method successfully reconstructs a 3D face model from a single RGB image and achieves state-of-the-art performance in the Normalized Mean Error(NME)of 3D facial landmarks.Further experiments on Celab A HQ dataset without 3D annotations confirm the feasibility of the proposed method for the wild images.
Keywords/Search Tags:Head Pose Estimation, Graph Convolutional Network, Multi-Layer Perceptron, 3D Point Cloud, Face Reconstruction, Facial Landmarks, 3D Morphable Model, Deep Learning, Convolutional Neural Networks
PDF Full Text Request
Related items