Research And Implementation Of Emotion Recognition Algorithm Based On Fusion Of Voice And Facial Expression

As an important part of artificial intelligence,automatic emotion recognition attracts many scholars to do related researches,mainly including the extraction of emotion features and the improvement of recognition algorithms.This paper intend to explores automatic emotion analysis.There are three parts in this research:facial expression recognition,emotion recognition based on voice features and multimodal emotion recognition based on facial expression and voice features.The first part focused on developing facial expression recognition algorithm.In this process,cubic spline interpolation coefficients,which proposed as geometric feature,and the HOG feature are adopted to represent the expression information.The support vector machine with Radial Basis Function kernel and the multi-layer perceptron using backpropagation are trained as classifiers.CK+ database and the JAFFE database are used to evaluate this system,in which feature layer fusion and decision layer fusion are applied.The results verify the effectiveness of this proposed feature,and also show that the setting of the hidden layer and learning rate of the neural network will affect the accuracy of recognition.Variation and size of the library and the diversity between people will also affect results.In addition,those two fusion methods applied in this process tend to have different results,due to different classifiers and databases.The second research is emotion recognition based on voice features.OpenSMILE is used to extract acoustic features with 1582 dimension,including sound quality features,prosody features and spectral characteristics.In those experiments,support vector machine with Radial Basis Function kernel is trained as the classifier,and CHEAVD 2.0 database,issued by Chinese Academy of Sciences,is used to evaluate this method.The accuracy of experiment is lower than expected,because those data are huge and complicated.It is hard to train SVM as classifier that can make valid prediction with voice features extracted from this database.The final study is multimodal emotion recognition using acoustic and image information,in which geometric features proposed in first study and the acoustic features extracted from speech are adopted in experiment.Additionally,feature layer fusion and decision layer fusion are adopted here to combine features from different modal.The CHEAVD 2.0 database is used in experiment,and support vector machine with RBF kernel is trained as classifier.The experiments results and corresponding analysis are demonstrated in this part.