The Design And Implementation Of Appearance-based Head Pose-free Gaze Estimation System

Gaze estimation is the process of predicting the direction of human sight.Gaze estimation has great research value and application value,and is widely used in human-computer interaction,virtual reality,assisted driving.There are mainly two gaze estimation methods,which are model-based and appearance-based gaze estimation.In model-based gaze estimation,it must have multiple infrared sources and high definition cameras,which limit the application area severely.The method of appearance-based gaze estimation has become the mainstream research method because of its simple equipment and wide application range.In this paper,we adopt the appearance-based gaze estimation which is a purely learning problem from a large number of data,mainly divided into training and testing phase.At present,there are mainly two problems in the apparent line-of-sight estimation method: one is the accuracy is low,the other is that it cannot adapt to large-scale head movement.In this paper,the human eye image and head pose are taken as the input feature and the gaze as the output feature.We choose the deep learning method to learn the regression function from the eye image and the head pose to the gaze.Based on the system calibration(including Zhang Zhengyou calibration and the camera pose estimation based on specular reflection),the human face and eye are detected by Viola-Jones object detection algorithm,and the facial feature points are detected by Supervised Descent Method(SDM),and the head pose is estimated by the 3D face model morphing based on the feature points of the face,and the gaze estimation model is learned by convolutional neural network(CNN).This paper implements an efficient data collection program and a real-time gaze estimation system.In this paper,63200 pairs of training data were collected,including left and right images,head pose and gaze direction.In head pose estimation,when the face feature points are invisible,coarse head pose estimation is added,which increases the amplitude of head movement.In pupil center detection,the paper adds a series of prior knowledge and post-processing process,can be robust to avoid the eyebrows,glasses,hair impact.In this paper,we improved the net structure of LeNet5 and GoogLeNet so that the classification network can be used regression task.We trained three models: the gaze estimation model is trained and tested on the MPII public datasets and the datasets collected by me respectively,the gaze point model is trained on the datasets collected by me.The experiments show that the average error of the gaze estimation model based on convolutional neural network is 5.2 ° in MPII dataset,average error in my dataset is 2.6° and the average error of gaze point estimation model is 220 pixels which improves the accuracy of gaze estimation while adapting to a wide range of head movement(-45 degree to +45 degree).