Research On Gaze Estimation Based On One Monocular Camera

Gaze estimation is one of the hot research topics in computer vision and patter recognition. It is very significant in the theoritic and practical aspects. Progress in gaze estimation could push these fields forward. Gaze estimation can also be used in Human-Compter Interaction (HCI), and psychology research. Although intrusive gaze estimation has made a big progress in recent years, non-intrusive gaze estimagtion is still in preliminary stage for application. To achieve robust non-intrusive gaze tracking system, it still needs to overcome some key problems. Especially, it needs effective feature and gaze estimation method to implement head-free gaze estimation.This thesis focuses on the some problems related to non-intrusive gaze estimation from a monocular camera. The problems include data collection, and automatically labeling the ground truth of the collected data, eye appreance feature representation, and head-free gaze estimation. The main contributions of the thesis are as following:1. Propose a data collection method which can capture gaze direction, head pose, and face image simultaneously, and a capture studio is implemented based on the above method. For a statistical learning algorithm, the performance relies on large amounts of labelled data. Therefore, the labeled data is the foundation of gaze estimation research. This thesis proposes a novel method of data collection in the complex environment. Our method can synchronously collect the images, head pose, gaze, and the spatial position of subjects. The collected data provides a guarantee for the further experimental training and testing.2. Propose a novel feature named Directional Binary Pattern (DBP) for gaze estimation. The sclera and the iris change their position within an eye socket with the change of gazing different directions. The change can be looked as horizontal and vertical movement of iris, which causes the texture change of eye image. To characterize iris vertical and horizontal movement, a directional binary pattern is proposed. By calculating the difference in the four directions, DBP not only contains the local texture information, but also contains specific directions binary differential information. Therefore, DBP is suitable to descript the texture changement of eye image related to the movement of iris. Mean while, DBP is robust to light variances and can decrease calculating error related to the light reflection.3. Propose a hybrid feature-based method for gaze estimation. Hybrid feature contains the model-based feature and appearance-based feature. Model-based feature contains the geometric vector among the feature points; appearance -based feature is extracted from the eye image based on the Gabor Directional Binary Pattern (GDBP). In this thesis, the combination of features is calculated by Support Vector Regression (SVR) algorithm and one of hybrid features corresponds to a gaze direction in a fixed head pose. For the appearance-based feature, the DBP operator successfully combines with the Gabor amplitude information, which has made a perfect performance. Hybrid feature-based approach has the following characteristics: (1) Binarize the eye image into different calculating directions. (2) Successful combination of the DBP operator and the Gabor amplitude informations, and the final discriminating feature is the extracted spatial histogram from the hybrid features. (3) Explode their statistical properties of features, and also benefit from the robustness to light variances.4. Propose a gaze estimation method which independs to head pose. To video-based gaze estimation, there are two important components: the head pose and gaze direction. At present, the algorithms realize the gaze tracking under the free head motion by calculating the head pose and gaze direction in sequence. This paper presents a distributed framework to estimate the head pose and gaze direction respectively, which can achieve the gaze tracking under the free head pose. On this basis, this paper proposes an algorithm for gaze tracking by the combination of the head and eye features. Experimental results show that our method is effective.In conclusion, through above-mentioned work, this dissertation makes a deep research on the problems of gaze estimation from a monocular camera. The experimental results show that the appearance-based feature and model-based feature have the discriminating information related to the same gaze direction. And then, excellent system performance can be acheved by effectively combining the two features. Moreover, in this dissertation, the gaze direction can be estimated by only one camera under the free head motion. And the proposed methods are applied in the gaze estimation system. The experimental results show that the proposed methods have the practical value.