Deep Learning Based 3D Face Attribute Recognition

Human faces provide us with not only identity,but also demographic attributes like gender,ethnicity,age and so on.Such demographic information,also known as soft biometrics,plays a very important role in many applications,e.g.,large scale face recognition,video surveillance,and social entertainment.A lot of remarkable achievements have been made in facial attribute recognition using 2D images especially since the employment of deep learning(DL)technology.Despite this phenomenal performance and availability of data,2D face recognition is challenged by changes in illumination,pose and scale.On the other hand,3D face recognition has the potential to address these shortcomings,and according to anatomical studies,the geometrical features of 3D face also contain abundant information of soft biometrics.However,it is still unknown about the effectiveness of DL in facial attributes analysis using 3D data.This thesis systematically investigates the performance of DL-based 3D face gender and ethnicity recognition with projection based approach and point cloud input based approach.Five representations including point clouds,depth images,normal maps,depth encoder map and depth-normal maps are compared for 3D facial attribute recognition on two benchmark databases,FRGC v2.0 and BU3D-FE.The main research work in this thesis is as follows:1.The original data of FRGC v2.0,BU3D-FE contain redundant information such as clothes and necks,as well as interference factors such as spikes.Therefore,we preprocess these 3D face data by nose-tip location,face clipping,denoising and so on.2.Referring to the data enhancement method of 2D face attribute recognition,the 3D face is rotated,adding random noise,rescaling in 3D space.Besides,100000 3D face models are synthesized by BFM data set to improve the performance and anti over-fitting ability of the model.3.In order to apply the Convolutional Neural Networks model directly,we use a variety of methods to project the three-dimensional point cloud to the two-dimensional space,and build a lightweight face attribute recognition network,to verify the effectiveness of these projection method and the generalization of the model through comparative experiments,ablation study,and cross-database experiments.Experiments on FRGC v2.0 show that the proposed methods can surpass the performance of existing work on gender recognition and race recognition.4.Using the existing point cloud classification model,the recognition rate is improved by improving the point cloud sampling method.The synthetic 3D face is used for model pretraining,and then FRGC v2.0 is used for fine-tuning to further improve the performance of the model.