The Applied Research Of Probabilistic Graphic Model On Affective Computing

As the development of personalized human computer interaction, more and more concerns have been put into the research on affective human computer inter-action in the fields of academics and industry. Against this backdrop, Affective Computing was raised by Prof. Rosalind Picard from MIT Media lab in1995, which has been widely applied in the fields of medical care, intelligent robot, elec-tronic commerce, security and so on. Though affective computing research has wide range of applications, many opening problems still exist. Due to the uncer-tainty of emotions, traditional machine learning algorithms may not handle this uncertain inference problem well. While probability theory provides us with the basic foundation to model the uncertainty of emotions. Although probability the-ory has been in existence since the17th century, our ability to use it effectively is fairly recent, due to the development of Probabilistic Graphical Models (PGMs), which include Bayesian Networks(BN), Hidden Markov Models(HMMs) etc. This thesis presents the research focusing on the applications of PGMs in some key issues of affective computing, which are summarized as following:Firstly, a large scaled visible/infrared multi-modal spontaneous/posed facial expression database, named USTC-NVIE database, is designed and constructed for the research of multi-modal based spontaneous/posed expression recognition, which contains both spontaneous and posed expressions of more than100subjects, recorded simultaneously by a visible and an infrared camera, with illumination provided from three different directions. Some statistical analyses are conducted to validate the effectiveness of the spontaneous expression eliciting videos, the relations of the annotated facial expressions and inner emotions, as well as the effectiveness of the infrared thermal images in the research of spontaneous expres-sion recognition and posed/spontaneous expression distignuishment. Secondly, due to the fact that most previous research on expression recogni-tion using thermal images are conducted on single expression image, some valuable timing information is lost, in addition. Thus, a spontaneous expression recogni-tion method based on temperature sequences of facial region is proposed at first, in which, the HMMs are utilized as classifiers to model the temperature sequence variance of different facial subregions. In addition, a posed/sponsaneous expres-sion recognition method based on BN and thermal images is firstly proposed.Thirdly, two implicit video emotion tagging method based on PGMs are pro-posed. In the traditional implicit emotion tagging research, the video emotions are always regarded as the audience’s facial expressions, which omit the rela-tions among facial expressions, inner emotions and video’s emotion tags. Thus, an implicit video emotion tagging method based BN by exploiting the relations among expressions, inner emotions and video emotion tags is proposed at first. In addition, due to the fact that some co-exist and mutual exclusion relations of multi-emotions are appeared, thus, another implicit video multi-emotion tag-ging method is proposed, in which, the relations among expressions as well as the relations between expressions and video emotions are modeled through BNs.Finally, action unit (AU) recognition based on incomplete data and PGM is proposed. Compared to expression annotation using some expression categories, AUs are more detailed for the understanding of expressions. However, the intensity of AU is ambiguous and professional knowledge are necessary for AU annotation, the AU annotation data is incomplete in many cases. Besides, there are some co-exist and mutual relations relations among AUs. Thus, an AU recognition method based on incomplete training data and PGM is proposed, in which, the expression is regarded as hidden information, and the relations between expressions and AUs are captured by BNs, the Expectation-Maximization (EM) algorithm is used to learn the parameters of the BN models under incomplete training data.