Cross-database Facial Expression Recognition Via Unsupervised Transfer Learning

Facial expression plays a vital role in human emotional expression and information transmission.If the computer has the ability of understanding and expressing emotions and further judge human's requirements,moods and hobbies according to expressions,then the computer will supply better services for human beings.This is of great theoretical significance and practical value in the fields of human-computer interaction,security,robot manufacturing,medical treatment,communication and automobile.With the rapid development of artificial intelligence and computer science technology,facial expression recognition(FER)has already attracted interesting results.It is notable that among the evaluation of the existing FER methods,the training and testing samples usually belong to the same facial expression database and hence the training and testing samples can be thought to share the same or similar feature distribution.However,in many practical scenarios,the training and testing facial expression samples may come from two different databases.In this case,the original similar feature distribution between the training and testing facial expression samples would not be satisfied.This thus brings us a challenging but interesting topic,i.e.,unsupervised cross-database FER whose label information provided by the test samples is completely unknown.The contributions of this thesis can be summarized from the following aspects:(1)A key frame extraction method is proposed in this thesis.By comparing the LBP histogram similarity between each frame image and the template frame,an appropriate threshold is selected to extract the key frames which solve the problem that excessive redundant information will lead to high space complexity of the algorithm.Key frame extraction is carried out in four databases: CK+,Oulu-CASIA VIS,Oulu-CASIA NIR and eNTERFACE,and a series of experiments have been done using key frame data sets.Affine transformation is used for face alignment as the preprocessing procedure before facial expression feature extraction and recognition.(2)A domain adaptive subspace learning(DoSL)model is proposed in chapter four.The basic idea of DoSL is to learn a projection matrix which transforms the source and target signals from the original feature space to a common subspace.In such common space,the source and target signals are enforced to obey the similar feature distributions.Hence we can train a classifier,based on the labeled source facial expression signals which could accurately predict the emotional states of the target facial expression signals.(3)A super wide regression network(SWiRN)is proposed in this thesis.Utilizing the powerful nonlinear representation abilities of deep learning networks,this model transforms the linear subspace model DoSL into the non-linear neural network model to deal with facial expression recognition.In this model,two fully connected networks are employed to elevate the sample feature into a super-high dimension space.Simply speaking,we use the network to serve as the regression parameter instead of the original parameter in linear subspace model in order to establish the relationship between facial expression features and labels,which project the sample features into label space.Regularization term is used to force the output features of SWiRN model for approximating the same or similar feature distribution.Finally,we can use this SWiRN to obtain the predicted label vector with the target facial expression sample as input and further infer its corresponding facial expression category.(4)A deep adaptative regression network(DARN)is proposed in chaper six.Based on SWiRN,an unsupervised facial expression recognition using convolutional neural network is proposed.The input of this model is raw images from the training testing samples,which does not need to extract handcraft features anymore.The whole network utilizes convolutional layers to extract features,and further restrict the similarity of the extract features by fully connected layers.Hence,the proposed model achieves an end-to-end operation.The DARN can fully use the advantages of big data,and overcome the vulnerability to illumination change,while also improve the insufficient robustness in real scenes.