Human-Computer Interaction Based On Eye Contact Detection

Eye contact is one of the common ways of communication in human social communication.At present,a large number of researchers have applied eye contact in practice,such as eye tracking,driver distraction detection,autism detection and so on.It is a challenging and valuable study to use computer to recognize human eyes.However,the existing methods are still constrained by many environmental factors,such as distance,head posture and illumination.For this reason,this paper proposes a method of gaze estimation based on convolution neural network,and implements a robust eye contact detection system.The eye contact detection system does not need additional equipment or manual calibration.The training process of this system mainly consists of four key phases: acquiring face image,data augmentation,regression of gaze direction and eye contact detection.There are two learning models in this paper,the visual direction regression model(VHModel)and the binary eye contact detection model,both of which are trained on the Colombia gaze data set.The VHModel is based on the resnet-50 training.During the training,the pre-processed eye image is used as input,and two key parameters representing the line of sight are obtained through training regression: vertical gaze direction V and horizontal gaze direction H.Then,we input the obtained V and H and the corresponding labels of eye contact into the binary classifier based on random forest,and learn to judge whether there is eye contact.The evaluation results of this paper show that our MCC value in the Columbia test set is up to 0.92,which is much higher than other comparative approaches.We also set up a balanced eye contact data set of positive and negative samples to further verify the generalization ability,and the data set contains about 2,200 face images in unfixed scenes.In order to better evaluate the robustness of the system,there are many differences of individual,background and posture in the image collecting,such as whether to wear glasses,distance change,head posture change,etc.Then,the accuracy of the system in this paper on this self-built data set is as high as 0.80,and the F1 score reaches 0.77.Finally,combining with the actual needs,we successfully enable the robot to receive the eye commands and make a series of responses,and also implement a trigger that can take photos according to the eyes.The main contributions of this paper are as follows.Firstly,we propose a new eye contact detection method based on convolutional neural network.The system trained by this method performs better than other existing methods,and is robust to brightness,distance and head posture.Secondly,in order to verify the generalization ability of the system,we also set up a binary eye detection data set.This data set is collected in unconstrained environments and could be used for the evaluation of other related studies.Finally,this paper combines the system with the practice to realize an application of human-robot interaction based on eye contact and a photo-taking trigger.