Emotion Recognition Based On The Fusion Of Expression Feature And Audio Feature

Emotion recognition plays an important role in the process of people's communication.In the field of human-computer interaction,the goal of emotion recognition is to make the computer have the ability of "perception" to realize the more natural and friendly interaction.At the first time,the majority of these works either focus on speech alone or expression only.With the deepening of research,people found that recognition of single mode has the limitation and the complementary relationship of these two modalities will help to improve the recognition performance.Because the audio and expression data will convey more emotion information,this paper focuses on emotion recognition based on facial expression feature and speech features fusion.For the first time,we apply the mixed probability canonical correlation analysis of MPCCA application in fusion features of facial expression and speech feature.Then we take the linear discriminant analysis thought into MPCCA and put forward the MDPCCA method,making up the shortage of CCA algorithm which can only solve the linear problem,at the same time increasing the difference between the different samples to improve the emotion recognition effect.In this paper,we first extract the facial expression feature,which is based on the BOW model improved by LLC coding way.We select the optimal parameters in the model through simulation experiments.MFCC and LPCC features are fused as the speech emotion feature.Then we analyze the result of emotion recognition based on single facial expression and speech features.Secondly,considering the internal relations of facial expression and speech features,we propose the partial canonical correlation analysis as the feature fusion;the algorithm can effectively improve the expression recognition rate of multi modal.Aiming at the limitations of the CCA algorithm in terms of fusion,MPCCA is used to fuse two multiple features from the probability density estimation model.Then we take the linear discriminant analysis thought into MPCCA and put forward the MDPCCA method.Finally,in order to verify the fusion algorithm proposed in this paper,we will compare the different fusion methods in the SAVEE dataset.The fusion methods include serial fusion,canonical correlation analysis CCA,mixture of probabilistic canonical correlation analysis MPCCA and mixture of discriminant probabilistic canonical correlation analysis MDPCCA.We verify the validity of the algorithm through comparative analysis of the different fusion methods results about the overall emotion recognition and single emotion recognition and design the emotion system based on MATLAB GUI.