The Research And Implementation Of Real-time Eye Tracking System

With the study of human eyes and gaze movements,the eye tracking technology has been steadily developed and applied to many fields through utilizing the increasingly advanced computer technology and artificial intelligence technology.However,there are still many problems in the eye tracking technology:harsh interaction causes poor experience in user,and poor performance on real-time,robustness and accuracy,which can't satisfy real situation.In this paper,the challenges of eye tracking technology are studied and analyzed,then chooses pupil corneal reflex and the method based on 3D eye model to achieve a new desktop eye tracking system.This paper presents three new algorithms to improve the performance of real-time eye tracking system,including pupil location algorithm,glint detection algorithm and pupil detection algorithm.The pupil location algorithm integrates traditional algorithm and deep learning,the grayscale characteristics of the pupil area can be used to process preliminarily the image and record the candidate region,then the pupil area can be detected by classifier;the glint detection algorithm adopts image processing algorithm to do threshold segmentation for pupil area,then it combines location characteristics of light emitters to solve the center of spot;the center of pupil can be calculated by pupil detection algorithm which contains morphological operator and ellipse fitting method,using Inpainting algorithm to avoid some interference factor,such as glare.Pupil localization based on object detection and pupil detection based on dark pupil can achieve real-time eye tracking system which has high accuracy and robustness.In this paper,a experiment database is established which gathered image data from 18 candidates,fully considering the real situation,such as illumination variation,head pose and motion blur.The performance of three algorithms in process of gaze feature detection is verified through locating and labelling pupil center and spot center.The results show that the accuracy of pupil location algorithm is 97.89%(the accuracy of classifier is 99.94%),the accuracy of glint detection algorithm is 98.7%,and the accuracy of pupil detection algorithm is 98.94%.A new real-time eye tracking system is presented in this paper,and a hardware architecture and software architecture also are proposed.Hardware consists of infrared source emitter,CMOS image sensor,FPGA data processing board and USB3.0 transmission board,it is mainly used to gather data.Then,module of software is achieved by QT framework and Opencv algorithm library,including algorithm module.It contains interface interaction and algorithm processing.The module of software have portability and could implement cross-platform development.The system contains four continuous processes:firstly,gathering image data with hardware device.The sensors capture dark pupil image,using FPGA to manipulation data and transform it to module of software by USB3.0.Secondly,image data can be processed by gaze feature detection to solve pupil center and spot center.Thirdly,solving optical axis vector with gaze estimation.Calculating camera internal reference and external reference with camera calibration,which adopt built-in calibration algorithm of Matlab.Center of curvature of corneal C and pupil center P,which is two parameter of optical axis vector,can be calculated by the system principle of single camera with double light source.Finally,Solving spot coordinates by deviation calibration.There is deflection in optical axis vector and visual axis vector,therefore this paper researches a spot averaging method and accuracy error method,which are established on four point calibration algorithm,can calculate the spot coordinate accurately.The result indicates the frame rate is 54fps,the accuracy is 0.67~o,it also has strong capacity of resisting disturbance,and well robustness,which could have a significant performance in real application scenarios.