A Study Of Emotion Recognition From Electrocardiography Signals

Affective Computing is a highly integrated research area that has rapidly evolved recently, aiming at giving the computer more emotional abilities in identifying, understanding and adapting for building a harmonious man-machine environment and further possessing a higher and comprehensive intelligence. As an important part of affective computing, emotion recognition mainly contains diverse studies including voice signals, body posture, facial expressions and physiological signals and so on. Owing to the characteristics of truth, credibility, and difficulties to counterfeit, it is becoming one of the hottest research fields based on physiological signals. Affective Computing Research Group from MIT Media Lab first has a research on emotion recognition from physiological signals, and had already reached some good achievement. Therefore, it also provides a reliable support for emotion recognition from physiological signals.As ECG and heart rate of abundant emotional features, they can clearly reflect human's emotional states under the different changes. Therefore, this paper has a research for emotion recognition and validation of user model from the extracted data on ECG and heart rate (six emotion states: happy, anger, surprise, fear, sadness and disgust). Main contents are as follows: data acquisition, feature extraction, feature selection and classifier design.This paper has planed elaborate rules for guaranteeing ECG and heart rate containing some special characteristics of emotions. In the experiment, the program is just designed to ensure that our chose movie clips are enough effective to elicit subject's emotion, which also need subjects to record their own feelings after watching over. Besides, operator will also carefully observe everything from subjects by the camera hidden in somewhere, and momentarily give the according marks on capture software Superlab. Through U.S. Multi-Polygraph MP150 from Biopac, this paper acquires emotion data from 300 freshman from Southwest University and then sets up emotion database on ECG and heart rate. For the effectively rigid selection on film clips and subjects, it meets ideal requirement at some extent that the collected signals must contain certain emotion which includes happy, surprise, disgust, sadness, anger and fear. Feature extraction of emotion signals on ECG and heart rate is mainly depending on position detection of P-QRS-T wave, but there are also some difficulties in accurately detecting P-QRS-T wave with noise interference such as ECG baseline drift, etc. Wavelet transform has a good time and frequency localization, and could reflect local characteristics of signals in the time-frequency domain, so it is widely be used in image analysis, denoising and compressing, etc. In this paper, continuous wavelet transform is suitably adopted to ECG signal for 5-layer decomposition. According to frequency range of R-wave based on the first level of wavelet coefficients, R-wave position can be exactly detected, and then Q, S, P and T-wave locations would be easily found. Through automatically filtering signal intervals of lower signal-noise ratio, it can finally remove the high-frequency noise and smoothen effective signal by the geometric mean..As effect of emotion recognition will be disturbed by the extracted features of redundant and ineffective characteristics in ECG and heart rate, then it is necessary to select effective feature for classifying emotion. Feature selection is a combinatorial optimization problem and its computational complexity would be exponentially enhanced with the increasingly amount of dimension, so it requires effective search algorithms to solve this troubles. Discrete binary particle swarm optimization (BPSO) is an intelligent global optimization algorithm, which has been widely applied at present to combinatorial optimization problems, function optimization, signal processing, neural network training, data mining and data clustering and other applications with more advantages in speed computing, simple algorithm parameters and easily implementing. Besides, sequence backward selection algorithm (SBS) is also an effective search algorithm. Therefore, this paper will satisfactorily introduce BPSO, SBS to feature selection on ECG and heart rate to improve recognition rate of emotion states. For BPSO can be easily lead to local search bringing into premature convergence, this paper proposes two kinds of improvement strategies: an improved algorithm is based on neighborhood search method (IBPSO), and it can conveniently supply particle swarm more opportunities to jump out of the local optimum to the global optimum; the other perfectly introduce the genetic operations (crossover and mutation) into the BPSO, and it mainly increase the diversity of evolution population. Moreover, as fisher classifier of high efficiency and accuracy, it is just imported to feature selection with BPSO and SBS.Ultimately, the experimental results show that the features of heart rate are superior to ECG for emotion recognition, especially in fear, surprise. In both improved BPSO algorithm, the average recognition rate attained by the best feature combination in heart rate is nearly respectively 10% higher than in ECG. Furthermore, for the same training, test and validation set, the best combinations are exactly different. So there are greater differences in recognition rate for validation among them. In general, the average recognition rate for validation by IBPSO is higher or as much as GBPSO (in addition to happy), and the dimension of the selected feature subset is obviously less accordingly, which shows that the best feature combination is more suitable for affective user modeling. And for SBS, although the best feature combinations for test and validation are worse than the two improved BPSO above, it gladly gained a few features that could achieve the same effect as BPSO during emotion recognition for happy. And surprisingly, some selected features also mostly existing in BPSO. So it does show that the feature combination is suitable for affective user modeling in happy...