Human Emotion Recognition Using Fused 2D-Fractional Fourier Transform Features Based On CCA

Facial expression related processing by computer has been an attractive issue in the areas such as image processing, computer vision and pattern recognition, which brings people practical value and economic value in the field of video conference, digital family, artificial intelligence etc. The visual information is one of the most important indicators which can represent the human emotion properly, thus, on the basis of the existed methods of human emotion recognition, we propose an approach for recognizing human emotional state from fused 2D-FrFT features based on Canonical Correlation Analysis (CCA). This approach mainly based on the correlation between different orders in 2D-FrFT. First, the visual features are extracted by 2D-FrFT, and choose two orders which achieve the highest recognition rate for feature fusion through CCA. Then send the fused features into the multi-classifier based on SVM. The feasibility of the recognition approach we proposed has been tested and the experimental results sufficiently demonstrate the effectiveness of the proposed approach. Our main work is listed as followed:For the first time 2D-FrFT has been used as the tool of feature extraction in this paper. Under the same simulation environment we compared 2D-FrFT to Gabor wavelet, the simulation result proved that 2D-FrFT is a effective and feasible tool for feature extraction. From the simulation results we explore the relationship between the transform order of 2D-FrFT and the recognition rate, find two optical orders for the best performance as well. From the view of recognition rate and information theory, we find two optical orders for the feature fusion in the next step;An effectively multi-classifier scheme based on SVM is proposed, it adopts the structure of "pyramid upside down". Compared to the classic classifier such as FLDA, HMM, SVM etc with the same simulation environment. The simulation result shows that the multi-classifier scheme we proposed can control the classified progress easily, which helps to reduce the misjudgment rate, obtain more accurate and reliable decisions. A human emotion recognition system based on feature fusion and multi-classifier is proposed. We fused the two optical transform orders by CCA and compared to the traditional serial fusion method. The simulation result shows that after feature fusion the fused features become more and more distinctive, the average recognition rate achieves higher, the dimension of the fused features has been reduced and the system computational complexity has been simplified.