User Emotion Recognition Based On Smartphone Sensors

Currently,the smartphone has become a mobile computing platform of computing,storage and sensing.It becomes an essential equipment of communication and sensing in people's daily life.We can provide people with a wide range of personalized services through the smartphone's application usage patterns and people's daily behaviors.And recent studies show that people's emotions are closely related to daily behavior and smartphone usage patterns.The advantages of using smartphones emotion recognition is that people's emotions can be speculated in real time without affecting their daily use to better understand people's mental state and to promote better communication between each other.Therefore,emotion recognition based on smartphones has very important theoretical and practical value.In this paper,we study about the problem of how to recognize the user emotion based on smartphone data more really.With single data used in the previous studies,it cannot make a comprehensive response of user behavior patterns.So we collected fine-grained sensing data which could reflect user daily behavior fully from multiple dimensions based on smartphone,and then used multidimensional data feature fusion method and result fusion method,using a variety of classification methods and emotion classification model to recognize emotion.Work in this paper is mainly reflected in the following three aspects:(1)The first problem is how to collect data and choosing which kind of sensing data closely related to emotions to collect.Firstly,we designed a network questionnaire on how emotions affect people's behavior patterns as the pre-survey,and analyze the survey results as the basis for selecting the type of phone data.Then,we design and implement a real-time data collection tools for collect all kinds of sensing data which are closely related with the user's emotion,at the same time developed an interactive tool to record daily emotion for users timely.(2)With the issues of mobile data scale expansion and type complicated for multi-sensor data collection,we use different preproccess methods and feature extraction methods to different types of data,and make data normalization at last in order to maximize the extraction of sensor data's features which can reflect the user behavior patterns.In this part,we lay the foundation for emotion recognition with the methods of multi-dimensional data fusion and result fusion.(3)Finally,we launched a study of emotion recognition based on smartphone with the use of feature fusion method and result fusion method.In this experiment,we used six classification methods such as Support Vector Machine and Random Forest and two emotion classification models.Then we carried out contrast experiment with 14 volunteers' hybrid data and personal data respectively to recognized user emotion based on discrete emotion model and circumplex emotion model.The results showed that the multidimensional data feature fusion method we mentioned which could better recognize user emotion,among them personal data training the accuracy rate can reach 79.78%,and circumplex model of affect is better than discrete emotion model.