Research On Multi-modal Emotion Recognition Method Of The Elderly Based On Deep Learning

In order to let computers have the ability to understand,recognize,underst and and express human emotions,emotional computing technology has been de eply studied,enabling humans and computers to achieve mutual understanding and efficient interaction.However,human beings are a complex group.Human expression emotions have a variety of carriers,including expressions,speech,physiological signals,body postures,etc.Through the study of emotion recogni tion of multiple modalities,it can help computers better understand human bein gs and create Easier human-computer interaction environment.This paper studie s the establishment of multi-modal elderly emotional database,the extraction an d recognition of speech emotion features,the extraction of facial expression fea tures and emotion recognition.The main research work and results are summar ized as follows:1.From the current situation of emotional research,we can find that the emotional research for the elderly is not enough.At the same time,the emotio nal database established by researchers at home and abroad,the emotional expr ession object is almost young,and the modality is single.In response to this p roblem,this paper constructs a video emotion database,a phonetic emotion cor pus and a facial expression image library in a TV series called "The Empty N est",and expounds the construction method and process of the multimodal emo tion library.The experimental results show that it is reasonable and effective to construct a multi-modal old-age emotional database.2.Based on the self-built multi-modal emotion database,the self-encoding neural network is used for feature dimension reduction,and the support vector machine is selected as the classifier to study the speech emotion recognition.In this paper,Fourier coefficient(FP),dynamic features(first-order difference and second-order difference)and global features(maximum,minimum,mean,median and variance)are extracted respectively for a total of 1800 characteristic parameters.Then,the self-encoding neural network is used to reduce the features,and the dimensionality-reduced feature parameters are sent to the classifier for emotionrecognition.Experimental results show that the highest rate of emotional recognition is reduced to 800-dimensional features.3.In the face expression image library of the empty nest grandfather,this paper proposes a method of facial expression recognition for the elderly using convolutional neural network as a model,which avoids complex feature extracti on of images.On the empty nest grandfather face image library,convolution o peration is performed on the input facial expression image,and then the poolin g layer in the convolutional neural network model is used to reduce the featur e dimension.Finally,the Softmax classifier is used to select the output value.The category corresponding to the largest neuron is used as the classification r esult.Compared with multi-decision neural network and self-encoding neural ne twork,the experimental results show that the convolutional neural network mod el is better applied to the empty nest grandfather image database.