Emotion Recognition Via Passive Sensing

With the rapid development of science and technology,emotion recognition has at-tracted more and more researchers'attentions.Emotion comes from a psychological or physiological process that is triggered by an event or object that is related to an individ-ual's mental states or characteristics.Therefore,emotions are closely related to people's mental well-beings.Emotion recognition can help people regulate emotions,detect po-tential psychological problems and prevent in advance.Besides,artificial intelligence(AI)are dramatically improving our world,emotion recognition is a key technology in the human-computer interactions(HCIs).If the robot can understand or recognize humans',emotions and respond appropriately,it can be smarter and more humanized.Traditional emotion recognition usually includes two methods:active sensing and passive sensing.Emotion recognition based on active sensing relies on expensive ded-icated equipment or specialized deployment.Besides,it requires individuals' active participations,which is high-cost.Passive sensing has been favored by researchers for its low cost,large scale data,and non-intrusive advantages.Therefore,we recognize emotions based on passive sensing.We utilize two kinds of commonly used passive sensing methods:online social network sensing and smartphone sensing.Based on the preprocessed sensor data,we build machine learning models to recognize emotions.Furthermore,we explore the application scenarios of emotion recognition under differ-ent passive sensing methods.In terms of emotion recognition based on passive sensing via online social networks,since most existing works only consider single emotion de-tection problem and ignores the coexistence of multiple emotions.Therefore,we have innovatively proposed multiple emotions detection problems in online social networks and formalized them as a multi-label learning problem.Besides,we propose a frame-work to study happiness by quantifying happiness and influence in order to mine highly influential users.In terms of emotion recognition based on passive sensing via smart-phones,we attempt to use sensor data for compound emotion detection.Compound emotions refer to emotion vectors with multiple levels presented by users over a time period.Besides,we also innovatively propose the mood instability detection problem via the smartphone sensing.We propose a multi-view classification model to solve the mood instability detection problem.The main contributions of this paper are as follows:(1)About the users' emotions recognition in online social networks,we propose an emotion recognition model based on multi-label learning.By analyzing one anno-tated Twitter dataset,we found that emotion labels correlation,temporal correlation and social correlation respectively.We design a multi-label learning algorithm based on factor graph to incorporate these correlations into a united framework to solve mul-tiple emotions recognition problems in online social networks.Performance evaluation shows that the proposed factor graph model outperforms other baseline algorithms.(2)About studying users' happiness and influence among friends in online social networks,we first quantify users' happiness using their daily published texts on online social networks by introducing happiness score.Next we use personalized multiple lin-ear regression models to quantify the happiness influence among users in online social networks.Since each individual's influence is different,we propose a greedy algorithm to detect a group of highly influential emotion representatives.The detected emotion representatives can be used as features to predict the happiness scores of other users within whole online social network.(3)For the compound emotion recognition problem via smartphone passive sens-ing,we develop the MoodExplorer system and propose a compound emotion recog-nition model based on multi-label learning.We first collect data from 30 volunteers'smartphones using an APP running on the Android platform.Based on data analysis,we find that users' compound emotions are highly correlated with their smartphone usage patterns and smartphone sensor data.We design a feature extraction and feature selection algorithm to find the most significant features.We further use the factor graph model to consider the correlations between features and emotion labels and the correla-tions among emotion labels.Experiments show that MoodExplorer can recognize the users' compound emotions with an average exact match of 76.0%.(4)For the mood instability detection problem via smartphone passive sensing,we propose a machine learning model framework to detect mood instability automati-cally.We first collect emotion data and sensor data from smartphones of 68 volunteers.We then quantify the mood instability using time-series emotion data.Based on sen-sor data,we design a temporal-sensitive feature extraction method by constructing a two-layer long and short term memory(LSTM)network to extract features.Finally,we propose an attention mechanism based multi-view learning classification model to optimize learning performance by considering cross-domain knowledge from different sensors.Experiments show that the proposed multi-view learning model outperforms other baseline methods.