| With the development of Internet and artificial intelligence technology,intelligent audio,voice assistant and other products appear on the market.If the machine can achieve speech emotion recognition,it can provide a more friendly user experience.In addition,speech emotion recognition has a broad application prospect in the treatment of depression patients,online distance education and other aspects.With the extensive application of deep learning in the field of speech emotion recognition,the accuracy of speech emotion recognition has been effectively improved.However,there are still many difficulties in the research of speech emotion recognition.Emotional information is unevenly distributed on the speech signal.In this case,how to extract more effective features puts forward more stringent requirements on the network structure.At the same time,in most of the research of speech emotion recognition based on deep learning,the input of deep learning model is designed by hand.The way of feature extraction also affects the recognition rate.In view of the above situation,this paper focuses on the speech part which is more useful for emotion classification,and uses deep learning model to extract features directly from the original speech signal.Firstly,this paper starts from the unbalanced distribution of emotional information in speech,extracts the features of each moment through 1D RESNET,and then adds the adaptive pooling module.The adaptive pooling module links the speech context information to predict the weight of each time feature,takes the weighted average value of all time features as the final feature,and then sends it to the full connection layer for classification.The weight is obtained through network learning,which is used to adjust the contribution degree of different time features in the global features after fusion.At the same time,this paper not only predicts the weight value of each time feature from the forward sequence,but also obtains another set of weight values from the reverse sequence.The validity of the adaptive pooling module is verified by experiments.Secondly,this paper analyzes the strong tag form of speech.This paper improves the above model of predicting importance score based on context information.According to this strong tag situation,the key frame loss function is constructed,and the importance score is explicitly constrained,so that it tends to 0 or 1,which is more in line with the situation of strong tag with or without emotional information.At the same time,the center loss function isadded to reduce the distance between classes and increase the distance between classes.The final experimental results also show that the key frame loss function and the center loss function can improve the recognition accuracy of the model.Finally,this paper starts from the input of neural network.In the study of speech emotion recognition,the input of most deep learning models is hand-designed features.There may be information loss in the process of obtaining hand designed features,so this paper constructs and uses pyramid network to extract features directly from each frame of original speech signal.On the other hand,because there are manual design features about voice based on human prior knowledge design,it is widely used in voice related fields and has good performance.So in this paper,the feature extracted by pyramid network is fused with MFCC,and then the feature with stronger representation ability is obtained.Finally,experiments verify the validity of the features extracted by pyramid network and fusion. |
