Fuzzy Expression Recognition By Combining Fuzzy Rough Sets And Deep Convolutional Neural Networks

Deep convolutional neural networks(DCNNs)have achieved superb performance in image classification and other computer vision tasks,where the classes of images are Boolean(two-valued).Fuzzy facial expression classification tasks,where membership degrees are used to describe the intensities of emotions,are widely encountered in practice.It is of great interests to extend DCNNs to fuzzy facial expression classification,in order to achieve a robust facial expression recognition system.In this paper,we firstly present the development of facial expression recognition system,deep convolutional neural networks and feature evaluation criteria based on fuzzy rough sets theory,summarize the steps to realize the deep fuzzy expression classification system,and introduce the related works and preliminaries.This paper mainly focuses on the issues of training a DCNN for fuzzy facial expression classification.Based on the fuzzy rough sets theory,a Fuzzy Rough Convolutional Neural Network(FRCNN)model has been proposed,and evaluated on Jaffe and BU-3DFE datasets.The main works are as follows:1.In this paper,we introduce fuzzy rough sets to develop a Fuzzy Rough Convolutional Neural Network(FRCNN),as fuzzy rough sets form a suitable mathematical tool to characterize uncertainty of classification.Based on the kernelized fuzzy rough sets theory,we construct an optimization objective for training CNNs by minimizing fuzzy classification uncertainty,and present the definition and optimization of fuzzy rough loss.2.This paper applies FRCNN to fuzzy facial expression classification,and compares the proposed method with other feature extraction and learning techniques based on two classifiers,Algorithm Adaption k-Nearest-Neighbors(AA-k NN)and Softmax Regression(SR).Experimental results demonstrate that FRCNN achieves the best performance comparing with LBP,Image Net pretrained DCNN and end-to-end fine-tuning framework on Jaffe and BU-3DFE datasets.3.This paper uses the Multi-task Cascaded CNN for face detection,and then conducts fuzzy facial expression classification in the wild,based on the proposed FRCNN model.To improve the generalization ability of the proposed framework,adversarial transfer networks are used for fine-tuning,and the effective-ness is verified by experiments.