Facial Beauty Prediction Based On Multi-task Transfer Learning

Facial beauty prediction tends to study how to endow computers with the ability to judge and predict the beauty of human face.At present,there are some problems in facial beauty prediction,such as lacking training samples,unclear evaluation indicators,and over-fitting of training depth network.Multi-task transfer learning can utilize extraneous information from source domains and related auxiliary tasks to improve task performance.We propose a study of facial beauty prediction based on multi-task transfer learning.The main research contents are as follows:(1)In facial beauty prediction,single-task learning is a common method and the effective information of related tasks is ignored.However,multi-task learning can improve the performance of main task by using the auxiliary information of related tasks.Therefore,we adopt a multi-task learning facial beauty prediction model,which is based on VGG16,VGG19 and Resnet50 network.At the same time,we compare the results of different models in facial beauty prediction,and the results show that the auxiliary task of gender recognition shares shallow features for the representation of facial beauty prediction,and the accuracy of facial beauty prediction by multi-task learning is outperformed by single-task learning.(2)To solve the problem of few data samples from facial beauty prediction and overfitting of training depth network,we construct a multi-task transfer learning model,which combines facial beauty prediction and gender recognition and the model pre-trained on the Imagenet datasets.A Multi-input Multi-task Beauty Network named 2M BeautyNet is proposed to guarantee the multi-task transfer learning.In order to alleviate the pressure between the main task and the auxiliary,we use a weight-loss self-learning strategy to further improve the performance.After the model is trained,the random forest will replace the Softmax classifier to enhance the generalization of our network.Experiments on Large Scale Asia Facial Beauty Database(LSAFBD)and SCUT-FBP5500 Database show that our method has achieved good results in facial beauty prediction,and the accuracy has achieved 68.23%.The proposed 2M BeautyNet structure is also suitable for multiple input situations in different databases.(3)Knowledge Distillation can distill some knowledge of teacher model into student model,reducing model complexity and parameter quantity.Due to the large number of parameters of 2M BeautyNet,we combine multi-task transfer learning with knowledge distillation for face beauty prediction.Firstly,a multi-task teacher model and a multi-task student model are constructed,in which the teacher model is 2M BeautyNet with many parameters,while the student model is a shallow network of CNN-5,Vgg-11 and LightCNN-9 with few parameters.Secondly,we trained the teacher model and calculated its soft targets.Finally,knowledge distillation is carried out by combining the soft targets of the teacher model and the soft with hard targets of the student model.Experimental results show that the teacher model achieves an accuracy of 68.23% in the facial beauty prediction,hence its structure is more complex and the number of parameters is more than 14793 K.Although the classification accuracy of the multi-task student model by knowledge distillation is 67.39%,its structure is simple and the parameter is only 1366 K.