Research On Emotion Recognition Method Based On Multimodal Physiological Signals

It is the premise of human-computer interaction to give the computer accurate understanding of people's emotional state.Physiological signals are bioelectrical signals generated by the interaction of human organs,which can spontaneously reflect human's real innermost feelings.It is the goal of many researchers to improve the emotion recognition performance of physiological signals in different environments.According to the different individuals,incentive materials and application scenarios,the complexity and distribution differences of the collected physiological signal samples will seriously restrict the effect of physiological emotion recognition and the generalization ability of the model.In view of this,based on the characteristics of different physiological signals,this dissertation studies the traditional physiological signal recognition method and the emotion classification task under the framework of neural network in different implementation schemes,aiming to improve the performance of physiological signal emotion recognition system,and enhance the stability and universality of the system in multi person or multi environment applications.1)Aiming at the problem that feature selection and model recognition tasks are isolated in traditional physiological emotion recognition methods,a fusion multi index feature selection(XGB-FMIFS)algorithm is proposed based on extreme gradient boosting(XGBoost)principle.Firstly,from the perspective of node splitting times,gain and the number of covering samples,the algorithm comprehensively considers the important feature range of weight,gain and cover in the sample training process,and obtains the best feature subset iteratively according to the accuracy evaluation standard,which avoids the one sidedness of single index feature range selection to a certain extent,at the same time,the possibility of feature subset falling into local optimum is reduced.In order to make the feature subset have strong distinguishing ability,the linear kernel principal component analysis is introduced for feature processing,which can make the sample more linearly separable in high-dimensional space.Experimental results show that,compared with the traditional filtering and single index feature selection methods,XGB-FMIFS algorithm can achieve better performance improvement under equal band fusion standard(EBFS)combined with linear kernel principal component analysis.2)Considering the difference of different individuals' experience of emotional materials,based on the emotional approach avoidance trend effect and the asymmetry of brain function,from the perspective of time window,the asymmetry index(As I)was introduced and combined with two groups of EEG channel signals(Fp1,Fp2,Fz and AF3,AF4,Fz)measures the emotional degree carried by the window information of physiological samples,and then selects the window samples with strong emotional degree.In order to solve the problem of the poor temporal correlation between window samples after ASI screening and insufficient number of samples caused by and the inability to slide the window,combined with the learning advantages of the reservoir in the temporal samples,the internal plastic learning is carried out for all samples,so as to obtain the stable feature nodes of samples in the reservoir.Finally,the XGB-FMIFS algorithm is used and improved,and effective feature nodes are selected and used for emotion recognition.The experimental results show that this method can improve the performance of physiological recognition in the implementation scheme of subject all participation(SAP)and leave one subject verification(LOSO),and most of the screened windows are concentrated in the middle of the whole sample period,which has a certain guiding significance for future research.3)In order to solve the problem of poor effect for emotion recognition due to the large difference of physiological samples distribution between domains,a physiological emotion recognition method based on feature space and example collaborative transfer optimization is proposed.Firstly,in order to further enrich the feature information,the phase permutation transfer entropy is extracted by band crossing operation,then the tradaboost algorithm is used to obtain the target domain labels consistent with the ARPCA learning model.The feature space embedded representation is modified by iterative updating,so that the effective complementarity of feature adaptation and sample weight learning can be realized between channels,frequency bands and windows.Finally,the ARJST algorithm is embedded into the XGB-FMIFS algorithm framework to obtain the physiological feature subset.The experimental results show that the performance and feasibility of the proposed algorithm are verified in the implementation schemes of Leave Multiple Subjects Out(LMSO)and cross database verification(CDV),and the stability of emotion recognition interface is improved in complex environment.4)In order to further improve the performance of inter domain physiological emotion recognition and the adaptive ability of model learning,a semi supervised generative confrontation framework algorithm based on continuous label fusion is proposed.Firstly,the samples are mapped in the reservoir according to the sequence of time sequence,and the time sequence encapsulation method is designed to encapsulate the mapped samples into unit samples with echo state characteristics.Then,the discrete label continuation method is designed to obtain more real continuous label information,and the joint information representation of the feature and label is obtained by fusing with the source domain coding features through Hadamard product.At the same time,the generator is used to generate coding feature samples to enhance the learning of target domain samples.After continuous confrontation learning,joint distribution matching and updating between domains are realized,and finally feature invariant space with class discrimination is obtained in the encoder.Experimental results show that the method can improve the performance of classification and recognition in the case of large differences in sample distribution between domains,and realize the adaptive learning of the model to a certain extent,which further improves the generalization ability of the system.