Research On Key Techniques Of Heterogeneous Facial Expression Recognition

With the advent of modern cameras to capture facial data, face images can be captured in diverse spectral bands such as Visual (VIS), near infrared (NIR), thermal infrared (TIR), or as sizes of3D facial profile. These diverse image types that are formed as a result of diverse image formation characteristics are termed heterogeneous.The automatic facial expression recognition of such a heterogeneous face is of great importance in this modern society of advanced technology for a variety of reasons spanning from trivial matters like gaming to a more serious matters like health and securities of states. However the available facial expression recognition systems have little input to offer so far as these heterogeneous data are concerned. Almost all the available systems deal with one set of database images such as Jaffe or at best two sets of different databases of the same spectral band. More so, even with the available systems, there are several inherent challenges in terms of inaccurate recognitions, slow execution time and misclassification, that unfortunately leads to system failure. This makes the modern facial expression recognition technology lacks behind its key competitors like fingerprint recognition and retina recognition for very sensitive technologies. It is based on this system failure due to the failure or lack of the will power to deal with heterogeneous data, coupled with slow speed and unsatisfactory accuracies that this research was undertaken to address some of the pitfalls. Thus the main focus of this study is twofold; and that is increasing both recognition accuracy and speed in heterogeneous expressive faces. In due course, this dissertation offers contributions in four major ways. The procedures and the contributions in all the phases are briefly described as follows:(1) Ability to detect all skin types under heterogeneous illuminations:This is the first contribution of the research. We proposed a novel technique to detect the skin of all human types. The techniques used here are image normalization by Phong’s light model with free variableness to represent light intensities. The model’s parameters are fitted into the red, green and blue channels of the skin data. The sensor response to spectral light reflected by a surface is computed. Then we used a piecewise linear decision boundary in the Cb-Cr and hue (from HSV) plane through the Bayesian decision rule to compute the skin threshold values. By this threshold we applied the intersection and union rules to segment the skin. The method is improved by erosion and filtering. This method is able to detect the skin of all types including some heterogeneous images of different spectral bands. The results of the method are analyzed with several skin databases. A skin detection rate of95.2%was recognized in the Compaq Skin database. The execution speed is also very encouraging. Unlike most of the existing methods this method has high detection accuracy in Asians and dark skinned people.(2) Face detection by multilayer feed forward neural network (MFFNN):We proposed skin detection based on multilayer feed-forward neural network. This is the second contribution and it addresses the detection inaccuracies and slower execution time in face detection techniques. We proposed a novel skin detector based on multilayer feed-forward neural network. The facial features were extracted by Gabor filters and normalized by the discrete cosine transform (DCT). The Gabor features were robust against rotation and translation. The DCT normalization technique is robust to variations in facial geometry and increased illumination. It also facilitates the separation of image into spectral sub-bands of differing importance. This exceptional property makes it suitable to deal with heterogeneous data of different spectral bands. The features were then reduced in dimension by the application of Bessel down-sampling method. This method maintained the perceptual quality of the image. The images were then classified into faces and non faces by MFFNN. The MFFNN was trained by the back-propagation algorithm. The specific novelty here spans from the formulation of the training algorithm and the use of Bessel down-sampling in face detection. The average detection accuracy is96.5%; and the time it takes to process a320×240image size running on a2GHz Intel Pentium IV machine is0.0301s. This performance is statistically significant (p<0.05) and is far better than most of the existing methods. (3) Tracking the feature points of the expressive face in different head or facial orientations by the ellipsoidal model:This signifies the third contribution. We proposed feature tracking based on ellipsoidal model. The method models a procedure that chooses a set of features from the expressive face and tracks them robustly from one frame to the other and discards all the other features of the expressive face which are no more needed for tracking. The facial features which are the eyes, mouth, cheeks and their edges in the face represented by a contour are tracked from one frame to the other by using the brightness change constraint. The effect of this is that, the face image can be captured and extracted at all cameras’viewpoints.(4) Ada-AdaSVM multi-classification procedure for heterogeneous expression recognition:This is the fourth and most important contribution. Here we proposed a new facial expression classifier based on the speed advantage of AdaBoost and the high accuracy advantage of the support vector machine (SVM). The first section is a feature selection component based on AdaBoost (Ada). This technique selects the finest of weak classifiers, and boosts the weights on the examples to weight the errors. Then, the subsequent filter is selected as the one that provides the most excellent result on the errors of the preceding filter. The second part is a merger classification by AdaBoost and SVM (AdaSVM). The whole framework in this section is called Ada-AdaSVM. The database images were partitioned into training and testing by leave-one-out-cross-validation. The training and testing is reiterated10times with different randomly generated sets; thus we obtained the recognition rate of every expression and average recognition rate of all test samples. The trial was performed using tenfold cross-validation to obtain the average recognition rate. The expression recognition performance of the proposed method is compared with AdaSVM, AdaBoost and SVM. The performance of Ada-AdaSVM is much superior in both accuracy and speed.