Research Of Emotional State Detection From Touch-based Behavior On Touch-screen Devices

Touch-Screen devices which become a new form of HCI have been widely applied in our life in such a rapid development of Internet information age. Many applications are migrating from desktops to touch-based devices. As the user for the requirement of intelligent terminal interaction experience is increasing, we raise the question of whether touch behaviors reflect plays' emotion states. This study is not only a valuable evaluation index for the application developers of the mobile intelligent terminal, but also a personalized service for the applications.This thesis mainly starts point with touch behavior, discusses the feasibility of using touch behavior to recognize emotions with the background theory. Data collection experiment are designed based on the feasibility and features are defined for describing the touch behavior. Lastly, classifiers are built to recognize emotions automatically. Research works of this thesis are shown below:Firstly, this thesis explores the technology of data collection from touch behavior. A puzzle game based on Android IADS-2 is designed for the data collection. Features which can represent the touch behaviors are improved from the collected data, and interval time is proposed. Discriminant analysis is used for the correlation analysis of the features.Secondly, ReliefF feature selection algorithm and ANN, SVM classification algorithm are combined to build the ReliefF-ANN and Relief F-SVM classification models for automatically recognizing four emotions(relaxed, excited, irritated and boring), two levels of pleasure and two levels of arousal. The experiment results show that based on the extracted features we can more accurately identify emotional classification. At the same time, the performance of classification algorithms based on ReliefF feature selection algorithm are improved compared with the original classification algorithm.Finally, gender factor is taken into account for our study. Samples of features are analyzed under two kinds of factors in gender and emotion. The results show that length and pressure have the similar effect for emotion recognition under different gender. But the effect of speed and interval time is larger under different gender. Data has multi-label when considering gender, the traditional classification algorithm for single label cannot meet the requirement for classification. Multi-label classification algorithm is proposed for the data. Mulan, multi-label classification tool based on java is used for the study. The evaluation index has been proposed for the classification.The model of emotion recognition based on touch behaviors on intelligent mobile terminal is important to the subsequent application.