Research And Implementation Of Key Technology For Eye Tracking System

As a research hotspot at home and abroad for many years,eye tracking has shown broad application prospects in hybrid enhanced artificial intelligence,military affairs,virtu-al reality/augmented reality,human-computer interaction,medical diagnosis,and usability research.However,the existing eye tracking systems are still faced with many challenges,such as low reliability,close user cooperation required for parameter calibration,limited freedom for users'head movement,difficulty in accurate recognition of different eye move-ment forms(including saccades,fixation,and smooth pursuits),and high cost in domestic production.In view of those problems,this dissertation focuses on four important aspects of eye tracking,i.e.,robust eye feature extraction against pseudo iris attacks,calibration-free gaze estimation with large-scale head movement,highly accurate visual behavior recogni-tion,and low-cost domestic software and hardware implementation for eye tracking system.The specific research contents and contributions of this dissertation are as follows.(1)Existing eye tracking systems cannot extract eye features robustly with the inter-ference of changing illumination,reflected light spots,blinkings,and blur.Besides,in high security areas such as weapon system control,pseudo iris attacks are difficult to preven-t.Aimed at these two problems,an eye feature extraction algorithm robust against attacks is proposed.This algorithm consists of two parts,the lightweight anti-attack iris locat-ing algorithm(LAILNet)and the robust pupil and reflected spot feature extraction algo-rithm(LAILNet-PDGD).The lightweight anti-attack iris locating algorithm,LAILNet,can achieve robust and high-precision locating from data interfered by illumination changes,re-flected light spots,blinkings,and blurring,and can also effectively filter out false irises such as printed and artificial irises.The performance of LAILNet on two public data sets and the Intelligent Photoelectric Image Target Recognition and Tracking(IPITRT)data set in the dissertation is consistent with the current best algorithm MT-PAD in 2019.Additional-ly,the amounts of parameters and computation of LAILNet are only 1/24 of MT-PAD,and the processing time of LAILNet is 1/2 of MT-PAD.Without loss of accuracy,LAILNet is lightweight and,of real-time,and suitable for embedded low-power devices.On the basis of LAILNet,the LAILNet-PDGD algorithm achieves robust feature extraction for pupils and light spots.With interferences from pupil occlusion,hair,eyebrow,far and near focal length,illumination change,blur,light spots,spectacles,etc.,the accuracy of pupil feature extraction in LAILNet-PDGD is as high as 95.49,superior to the mainstream Deepeye and ElSe algorithms.It takes LAILNet-PDGD 5.76 milliseconds to process an 640×480 iris image,which is only 1/18 of the processing time of Deepeye.Robust anti-attack eye feature extraction algorithms will run efficiently on embedded,mobile,and low-power devices.(2)Though the 3-D eye model based gaze estimation method has advantages of high accuracy and free head movement allowed,the existing high-precision calibration-free gaze estimation method still needs a complex hardware configuration with dual cameras and four light sources.Aiming at this problem,this dissertation proposes a 3D binocular model based calibration-free gaze estimation method with one camera and dual light sources.In this method,a low-cost hardware configuration with one camera and dual light sources is adopted,the gaze estimation model is optimized,and the calibration-free gaze estimation is achieved under users'natural head movements.This estimation method is composed of two procedures,i.e.,automatic user parameter calibration and real time gaze estimation.Since real eye parameters,which varies from person to person,are required in real time gaze estimation,the dissertation at first considers the feature that the distance between the cornea curvature centers of users'two eyes should be fixed,and constructs a model to solve the cornea curvature radiuses of two eyes.Then the distance between the pupil center and the cornea curvature center is solved with pupillary boundary points.By the feature that the two eyes'visual axises gaze on the same point,the angle between the optical axis and the visual axis is calculated in the eye coordinate system.In the real time gaze calculating algorithm,at first a 3-D binocular eyeball model is constructed,the cornea curvature center and the pupil center are solved,and the optical axis is rebuilt according to geometric optic princi-ples.Then the fixed angle between the optical axis and the visual axis in the eye coordinate system is used to calculate the user's gaze.Finally,in order to verify the effectiveness of the automatic calibration algorithm and real time gaze estimation algorithm,the dissertation adopts the existing eye framework to simulate the binocular eye tracking process and obtain the experimental data and verification data of eye features.Based on these eye movement feature data,the automatic calibration algorithm and gaze estimation algorithm were tested to verify their effectiveness.The proposed calibration-free gaze estimation method based on the 3D binocular model with one single camera and dual light sources is simple in hard-ware structure,free of user calibration in the binocular eyeball model,and robust in head movement during eye tracking,which is superior to the single eyeball model as well as the binocular eyeball calibration model with one single camera and one light source under the highest accuracy and the most complex hardware structure.(3)Accurate eye movement pattern recognition based on eye movement data is the key to the application of eye tracking system.However,due to the factors such as noise,in-accuracy in the eye tracker,and the inherent eye movement features,the accuracy of eye movement pattern recognition is not high enough.Especially in smooth pursuit recogni-tion,the highest accuracy in 2019 is only 73%.Focusing on the problem that the existing smooth pursuit recognitions are low in accuracy and largely influenced by thresholds,the dissertation proposes a segmentation and clustering based recognition(I-SC)algorithm to identify three common eye movement forms,i.e.,fixations,saccades and smooth pursuit-s.First,the speed feature in eye movement data is used to identify the saccade segments.Second,the remaining eye movement data are divided into segments according to the stan-dard deviation of spatial features.At last,the mean direct distance feature is defined,and the clustering method based on fast peak density search is used to classify and recognize fixations and smooth pursuits.In the algorithm,the continuity and suddenness of eye move-ments are considered,the standard deviation of spatial features can reflect the suddenness of eye movements,while the continuity of eye movement is characterized by the segmentation method.The clustering algorithm implements thresholdless distinction between fixations and smooth pursuits.By using this method,accurate classification can be achieved even with noisy or inaccurate data from eye trackers.To prove the effectiveness and robustness,we evaluated the performance of I-SC with the data set sampled from 11 participants by a consumer-level eye tracker.Experimental results show that the recognition accuracy of I-SC algorithm is 96%,and its recall rate is 87.6%,which is better than the mainstream I-VDT al-gorithm and CNN algorithm,proving the I-SC algorithm can provide more accurate ternary classification.(4)Facing with the experimental equipment demands from theoretical research and applications and challenged by problems such as foreign eye tracking products are not avail-able in sensitive fields,high in price,and lacking in independent intellectual property rights,we designed and implemented an eye tracking system suitable for research and practical application.Our design is on the basis of the above-mentioned key technology research and long-term massive technical accumulation.In this system,the above attack-robust eye feature extraction algorithm,the 3D binocular model based calibration-free gaze estimation method,and the segmentation and clustering based pattern recognition(I-SC)algorithm are implemented.The designed eye tracking system is highly robust in anti-attack,in support of users' natural head movement,calibration-free,high in accuracy and low in cost.The implementation solution and indicators of the gaze tracking system are firstly designed,then the hardware and software frameworks are determined,and finally the testing and applica-tion of the gaze tracking system are implemented.Redundancy design is considered in the system hardware design.The low-cost dual-channel synchronous CMOS sensor is used in the hardware device to realize bright and dark pupil tracking with dual cameras and dual light sources,which can provide stable dual-channel bright/dark pupil or dark pupil image with 1280 × 720 resolution at 60 Hz.The entire hardware cost is within 1,000 RMB.In the software framework design,cross-platform QT software design is adopted,and a stable soft-ware framework environment is realized by using multi-thread and signal slot mechanisms.A standardized video acquisition interface is implemented for two different operating sys-tems Windows and Linux,so that the upper layer software of the eye tracking system can be compatible under different operating systems.Considering research and application re-quirements,the open iris database IPITRT was built.It is abundant in iris data variety with diverse interference and high background noise,and includes pseudo irises of high similari-ty with real ones.To verify the effectiveness of the hardware and software platform as well as the gaze tracking algorithm,we conducted a detailed test on 11 subjects under environ-ments separately with slight head movement,a wide range of head movement,and different illumination conditions,respectively,and then calculated the accuracy and precision of eye tracking.Experimental results show that the average accuracy of the eye tracking system constructed in this dissertation is 0.62°,the gaze tracking accuracy with a wide range of head movements and different illumination conditions is also about 1°,and the processing time of the eye tracking system is 9.67ms,which meets the requirements of both theoretical study and engineering real-time application.