Emotion Recognition Algorithm Research Based On The Speech And Facial Expression Information

Single-mode emotion recognition due to restrictions on the single modal characteristics of emotion recognition rate has not been greatly improved. In recent years, multi-modal emotion recognition of this limitation, the emotion recognition process, the introduction of a variety of modal integration of emotional characteristics, which have been greatly improved in the recognition rate.Currently, the methods of multimodal emotion recognition are mainly decision level fusion and feature level. In this paper, the feature level fusion approach, facial expression features extraction and speech emotion features, according to the characteristics of two kinds of mode of emotional feature, feature optimization, classification and classifier is designed to emotion. This paper choose this topic group self emotion database as the research data, the data in the database contains speech, emotion expression and EEG of three modes, emotional categories have7kinds, namely angry, disgust, fear, happy, neutral, sadness and surprise.The main research works of this paper are:(1) Extraction of speech emotion feature extraction method, the different pronunciation features (14dimensional and74dimensional feature), including short-time energy, pitch frequency, the first resonance peak, Mel frequency cepstrum coefficients are extracted (MFCC) and speech duration feature category, statistical parameters of these feature category related were also calculated, and based on these as the feature of speech emotion feature data for emotion recognition.(2) Facial expression features extraction, this paper proposes the texture feature LBP expression feature extraction algorithm for improving the extraction of face of the main two parts of the eyes and mouth. The algorithm is intended to ensure that the facial expression recognition rate, at the same time, to reduce the dimension of feature data as far as possible, reduce the amount of calculation.(3) The integration of voice and facial expression feature, according to the emotion characteristic expressions and speech, put forward the direct speech and expression feature fusion algorithm and speech and expression feature optimization algorithm. Direct fusion algorithm for speech and expression feature mainly solves two modal characteristic dimension of difference; speech and expression feature fusion algorithm considering the links and differences between the two modal characteristics, put forward first fusion, using the PCA method is adopted to reduce the dimension optimization, then the sentiment classification.(4) Bimodal emotion recognition, we use emotion recognition SVM algorithm simulation experiments. The algorithm for small samples, nonlinear classification problem has strong classification ability. In SVM parameter optimization problem, this paper proposes an improved grid search parameter optimization algorithm, the basic idea is to search through the rough basic grid search algorithm to determine the range of parameters, and then within this range for fine search, to find the optimal combination of parameters recognition rate. Simulation results show the effectiveness of the algorithm.