Research On Facial Attractiveness Prediction From 2D To 3D

Facial attractiveness is a complicated and inexhaustible topic.The attractiveness is obviously shown on the face,which can be perceived with little human effort,however,the determination of where it is remains a tought task.The pursuit of beautiful faces is human nature,inspiring researchers from multiple disciplines to analyze the components of facial attractiveness objectively and accurately.With the advent of visual age,face images are pervasive in all aspects of social life.Reliable quantification of facial attractiveness from a computational point of view not only leads to new methods for cross disciplinary studies,but also facilitates the development of many practical applications,thus having great scientific significance and broad application prospect.Integrating techniques from image processing,computer vision,and machine learning,this dissertation aims to investigate the emgerging research on facial attractiveness prediction,i.e.,to explore the salient elements of attractiveness from face data in two-dimensional to three-dimensional domain and develop humanlike beauty prediction models.The novelty and main contributions of this dissertation include:(1)An experimental database that aims for 2D to 3D facial attractiveness prediction is constructed.After the data processing and filtering,the original BJUT-3D database is pruned to 360 high quality faces,whereby 2D,2.5D,and 3D face databases are constructed.Through a well-designed attractiveness rating experiment and score verification process,all the rating scores of the 2.5D face database assigned by 48 reliable human raters are collected,as well as each rater's age,gender,and ethnicity,which finally constitute the attractiveness score database for three types of face databases.This new benchmark addresses the lack of available databases in 2.5D and 3D facial attractiveness applications,and also serves as the basis for the evaluation of attractiveness prediction approaches.(2)A 2D facial attractiveness prediction approach is proposed based on rule-driven and data-driven feature extraction manners.Inspired by facial aesthetic rules and commonly-used descriptors,a set of rule-driven geometric and appearance features is handcrafted.The idea of data-driven is employed to construct an exhaustive pool of candidate ratios,from which the most discriminative subset is chosen by an incremental feature selection algorithm.Experimental results show that the attractiveness prediction with such subset outperforms that with the rule-driven geometric features as wells as their combinations,reinforcing the significance of the data-driven method in 2D facial geometric feature extraction.The prediction results are further improved when rule-driven appearance descriptors are incorporated.(3)A 2D facial attractiveness prediction scheme by appling the label distribution learning paradigm is proposed,where a label distribution is treated as the ground-truth description of attractiveness of a face rather than the average score.The pre-trained deep residual network is transferred to 2D aesthetics-aware feature learning,and a three-layer neural-network-based label distribution model is also integrated into an end-to-end facial attractiveness prediction framework.Experimental results demonstrate the effectiveness of the residual network in learning high-level aesthetics face representations,and the advantages of label distribution learning over single-label learning in the attractiveness learning process.The fusion of deep features and low-level geometric features can further boost the prediction accuracy.(4)A data-driven geometric-based approach on 2.5D facial attractiveness prediction is proposed.Using the data-driven feature extraction and incremental feature selection,the most discriminative ratios,angles,and inclinations are obtained from the frontal and profile view of faces,respectively.A hybrid 2.5D attractiveness prediction model is developed via score level fusion and achieves superior performance to either view.The role of group categorical variables in facial attractiveness judgements is also explored.By incorporating them into the attractiveness modeling,an attractiveness prediction model is customized which can be adapted to the corresponding rater groups.Although such customized model performs worse than the universal model,it better matches the beauty perception among different groups.(5)A rule-driven and data-driven geometric-based approach is proposed for 3D facial attractiveness prediction.Using these two feature extraction methods,the aesthetics-aware geometric features are explored,including features directly extended from 2D task to the 3D space,i.e.,the shape indicator and Euclidean ratios,and features exclusively designed for 3D face structure,i.e.,the curvature descriptors and geodesic ratios.Experimental results demonstrate the effectiveness of the data-driven manner in 3D aesthetics representation extraction,the significance of 3D geometric features in capturing the face geometry over their 2D counterparts,and the advantages of the exclusive features in characterizing facial attractiveness over the extended features.