| To enhance the interaction between human and computer, a major task for the Human Computer Interaction (HCI) community is to equip the computer with the ability to recognize the user's affective states, intentions and needs in real time and in a non-intrusive manner. Using video cameras together with a set of computer vision techniques to interpret and understand the human's behaviors; vision-based human sensing technology has the advantages of non-intrusiveness and naturalness. Since the human face contains rich and powerful information about human behaviors, it has been extensively studied. Typical facial behaviors characterizing human states include eye gaze, head gestures and facial expression. This research focuses on developing real time and non-intrusive computer vision techniques to understand and recognize various facial behaviors.; Specifically, we have developed a range of computer vision techniques. First, based on systematically combining the appearance model with the bright-pupil effect of the eye, we develop a new real-time technique to robustly detect and track the eyes under variable lightings and face orientations. Second, we introduce a new gaze estimation method for robustly tracking eye-gaze under natural head movement and with minimum personal calibration. Third, a robust visual tracking framework is proposed to track the faces under significant changes in lighting, scale, facial expression and face movement. Fourth, given the detected face, we develop a new technique for detecting and tracking twenty-eight facial features under significant facial expressions and various face orientations. Fifth, based on the set of tracked facial features, a framework is proposed to recover the rigid and non-rigid facial motions successfully from a monocular image sequence. Subsequently, from the recovered non-rigid facial motions, a Dynamic Bayesian Network is utilized to model and recognize the six basic facial expressions under natural head movement.; All of these techniques are extensively tested with numerous subjects under various situations such as different lighting conditions, significant head movements, wearing glasses, etc. Experimental study shows significant improvement of our techniques over the existing techniques. |
