| Mobile terminals and smart devices are currently closely related to human daily life,learning and work.Affective computing based on smart devices has become a focus for domestic and foreign scholars.With the aggravate ageing problem,the needs of home care for elderly are increasing,and we can better understand and pay attention to the physical and mental health of the elderly by studying the emotional state and behavior posture of the elderly.A harmonious man-machine environment can be established through affective computing,However,there are still some problems that need to be solved urgently in affective computing.In human activity recognition,although the low-level statistical features of the cell phone sensor are usually used to recognize the human behavior,these low-level features ignore the high-level semantic expression of behaviors,which results a poor recognition rate of limited sample behavior in the training dataset.In emotion recognition,human emotions are composed of multimodal information,such as physiology,psychology,expression and tonality.When using one modality for emotion recognition,the poor emotion recognition rate is easily caused by lack of information for emotional expression.In view of the above two recognition problems,two recognition methods are proposed individually.The main work includes the following two aspects:(1)In order to solve the poor recognition performance of current fall detection caused by fall samples collected with difficulty and small size of fall samples in database,a human activity recognition method based on the low-level features and high-lever semantic was proposed.The semantic attributes between the low-features and the categories were introduced,which could be used to share the feature information of activities in the case of the activity had few samples.The attribute detectors were trained by the attribute-activity matrix and the low-features,and thus the attribute features were obtained.Then,the random forest algorithm was applied to recognize the attribute features and low-level features.Finally,the final result obtained by fusing the two modal pre-recognition results.The experimental results show that the proposed method has better classification performance compared with other methods.(2)Numerous emotion recognition approaches have been proposed,most of which focus on visual,acoustic or psycho-physiological information individually.Although more recent research has considered multimodal approaches,individual modalities are often combined only by simple fusion or are directly fused with deep learning networks at the feature level.In this paper,we propose an approach to training several specialist networks that employs deep learning techniques to fuse the features of individual modalities.This approach includes a multimodal deep belief network(MDBN),which optimizes and fuses unified psycho-physiological features derived from the features of multiple psycho-physiological signals(ECG,SCL,and tEMG);a bimodal deep belief network(BDBN)that focuses on representative visual features among the features of a video stream;and another BDBN that focuses on the high multimodal features in the unified features obtained from two modalities.Experiments conducted on the BioVid Emo DB database result in mean accuracies of 80.89%,which represents a considerable improvement over state-of-the-art approaches.The results demonstrate that this approach can solve the problems of feature redundancy and a lack of key features caused by multimodal fusion and improves the performance of multimodal emotion recognition. |
PDF Full Text Request