Remote, Non-Contact Gaze Estimation with Minimal Subject Cooperation

This thesis presents a novel system that estimates the point-of-gaze (where a person is looking at) remotely while allowing for free head movements and minimizing personal calibration requirements. The point-of-gaze is estimated from the pupil and corneal reflections (virtual images of infrared light sources that are formed by reflection on the front corneal surface, which acts as a convex mirror) extracted from eye images captured by video cameras. Based on the laws of geometrical optics, a detailed general mathematical model for point-of-gaze estimation using the pupil and corneal reflections is developed. Using this model, the full range of possible system configurations (from one camera and one light source to multiple cameras and light sources) is analyzed. This analysis shows that two cameras and two light sources is the simplest system configuration that can be used to reconstruct the optic axis of the eye in 3-D space, and therefore measure eye movements, without the need for personal calibration. To estimate the point-of-gaze, a simple single-point personal calibration procedure is needed. The performance of the point-of-gaze estimation depends on the geometrical arrangement of the cameras and light sources and the method used to reconstruct the optic axis of the eye. Using a comprehensive simulation framework developed from the mathematical model, the performance of several gaze estimation methods of varied complexity is investigated for different geometrical system setups in the presence of noise in the extracted eye features, deviation of the corneal shape from the ideal spherical shape and errors in system parameters. The results of this investigation indicate the method(s) and geometrical setup(s) that are optimal for different sets of conditions, thereby providing guidelines for system implementation. Experimental results with adults, obtained with a system that follows those guidelines, exhibit RMS point-of-gaze estimation errors of 0.4-0.6º of visual angle (comparable to the best commercially available systems, which require multiple-point personal calibration procedures). Preliminary results with infants demonstrate the ability of the proposed system to record infants' visual scanning patterns, enabling applications that are very difficult or impossible to carry out with previously existing technologies (e.g., study of infants' visual and oculomotor systems).