Deep Self-taught Learning For Facial Beauty Prediction

Facial beauty prediction aims to evaluate the beauty of different face images with different aesthetic features through machine learning methods, and produce human-like facial beauty intelligent perception. Geometric features and apparent features are the main aesthetic features in facial beauty prediction. The extraction process of former is involved with a great deal of artificial feature selections, leading to no authoritative results with too much interventions of active factors. Though apparent features are not confined to costly manual landmarks of facial features, simple information is just utilized to represent aesthetic features, such as edges and texture information. and is not involved in more hierarchical, structural and high-level feature express. The research of facial beauty belongs to a recently springing-up leading research topic, and a majority of face databases for facial beauty research are based on small samples and few of them are public at home and abroad. Therefore, the deficiency of a large number of training samples undoubtedly brings a realistic difficulty to research work. Deep self-taught learning simulates the architectural depth of brain, which automatically learns and extracts structural and multi-level features of samples without relying on artificial feature selection, and offsets the deficiency of the number of training samples, thus provides more scientific theoretical basis for facial beauty prediction. In this paper, deep self-taught learning is proposed for facial beauty prediction. The main work of this thesis includes the following aspects:(1) Self-taught learning is utilized to pretrain the network, avoid deficient training and improve the ability of learning beautiful information of face images. Self-taught learning aims to learn the features of unlabeled samples, such as nature images which do not need to share the class labels or the generative distribution of labeled samples, and use the learned features to characterize the features of labeled samples, thus effectively avoid the problem of insufficient number of training samples. Convolutional restricted Boltzmann machine (CRBM) is a hierarchical network, the apparent features from pooling layer not only take the advantage of decreasing feature dimension, but also allow representations translation-invariant with inputs. The use of self-taught learning to pretrain CRBM helps network to sufficiently learn the beautiful information of face images, thus benefits to the research of facial beauty prediction.(2) More suitable aesthetic features in facial beauty prediction are explored. Eigenface, texture features of Local Binary Pattern (LBP) and Gabor, and apparent features from CRBM are selected as aesthetic features, and support vector machine and K nearest neighbor are used as classifiers to evaluate the ability of those features in characterizing aesthetic information. Meanwhile, the performance of facial beauty prediction towards CRBM based on self-taught learning or not is conducted. Experimental results show that the extracted apparent features from CRBM combined with self-taught learning characterize more feature information of beautiful faces than the other aesthetic features, and are more helpful for facial beauty prediction.(3) More precise regression methods in facial beauty prediction are explored. The performances of standard linear regression, k-Nearest Neighbor (KNN) regression, multinomial logistic regression, ridge regression, and SVM regression for facial beauty prediction are evaluated, and the effects of different apparent features are also examined. Experimental results show that SVM works best in comparison with five regression methods and apparent features extracted by CRBM are more superior to describe experimental data.(4) Convolutional deep belief network combined with self-taught learning to form deep self-taught learning is proposed for learning beautiful concept and producing human-like predictor. Self-taught learning is used for unsupervised pretraining of convolutional deep belief network, which helps to provide good optimized starting point of deep network parameters. The performance of the representation from two-layer CDBN and single layer CDBN are compared when pixel-level images, LBP and Gabor features are treated as inputs of network respectively. Moreover, the performances of deep self-taught learning towards different image resolution under five regression models are evaluated. Experimental results show that when two-layer apparent features are concatenated and LBP texture features are treated as inputs of two-layer CDBN, the scores obtained from machine by SVM regression are highly consistent with those from labor. and further improvements are obtained with the increasing of image resolution.