Learning An Appearance-based Gaze Estimator From Large-scale Human Eye Image Synthesis

Gaze estimation is the process of calculating the direction of the gaze and the position of the fixation point,it has great potential in various fields.However,there are still many problems in the training process that require a large amount of marked data and redundant information.In order to reduce the cost of tagging data,the current approach is to replace the manually labeled real dataset with a synthetic dataset that can be automatically annotated.There is still a large gap in the distribution between the real image and the synthetic image.In order to reduce this gap,the previous method was to make the synthetic image learn the distribution of the real image to be more realistic when training a model.However,the drawback of this method is that the existing method only considers the global features while ignoring the local features,the real-time and automatic cannot meet the requirements.This paper focuses on the gaze estimation under large-scale data and the narrowing of the gap of the distribution between the synthetic image and the real image.The paper has made important progress in the following aspects:1.We propose a gaze estimation method based on neighborhood selection in large-scale synthetic images,which makes corresponding feature selections in head pose,pupil center and eye space.We propose a pupil center positioning method using image threshold and region clipping which can accurately detect the pupil center with a large head movement on a low-resolution eye image.2.We propose a gaze estimation method based on neighborhood selection in large-scale synthetic images to effectively reduce image distortion on the basis of retaining local information as much as possible,minimizing the need for actual data annotation.Furthermore,we draw on the design of the loss function in the idea of style transfer,increase the data penalty mechanism,and improve the discriminating effect.3.We propose a gaze estimation method based on the improvement of large-scale real images.The task-guided loss is calculated by the loss network to ensure that the distribution gap between the real image and the synthetic image is reduced.In summary,based on the research of gaze estimation under large-scale data and the reduction of the gap between the synthetic image and the real image distribution,based on the idea of neighbor selection,generative adversarial networks and style transfer,three different gaze estimation methods are proposed.Experiments show that the three gaze estimation methods can obtain the state-of-the-art accuracy on multiple public data sets.