Research On Eye Tracking Technology For Mobile Devices Based On Lightweight Networ

In recent years,with the development of computer performance and deep learning field,the field of vision detection has seen a spurt of development.As a more important branch of detection,the importance of line-of-sight tracking has been verified in practical applications,and the application of deep learning to line-of-sight tracking has become a mainstream research direction.However,deep learning-based line-of-sight tracking cannot be applied to mobile devices due to the complexity of its network structure.In this paper,we study the theory of lightweight networks,propose the method of line-of-sight tracking based on lightweight networks,and conduct experiments on line-of-sight estimation on mobile devices.The main work of this paper is as follows.(1)A lightweight 3D line-of-sight estimation algorithm is proposed.In this paper,the line-of-sight estimation is decomposed into 2 parts,head pose estimation and eye feature extraction,and a multi-loss head pose estimation network with Res Net V2 50 and Shufflenet V2 is proposed through the study of head pose estimation network,and the face alignment method is introduced to compensate for the head pose estimation.The eye feature extraction network with Shufflenet V2 algorithm is proposed,and the attention mechanism is introduced to improve the eye gaze point estimation accuracy.(2)Lightweight gaze point estimation algorithm is proposed.In this paper,the gaze point estimation algorithm based on Shufflenet V2 algorithm is proposed and tested directly using the dataset collected from cell phones in order to restore the real scene.The output of gaze point coordinates is performed by inputting face,human eye picture and face position information into the network and combining 2 fully connected layers.The effects of different face information input and different learning rates on model training are also investigated.(3)Algorithm porting on mobile,by converting the gaze point estimation model into tflite format and combining android Studio,Java and other technologies to design on mobile APP.In order to examine the accuracy,robustness,and interference resistance of the algorithm an experimental study of the gaze point estimation on the mobile phone side was conducted.