Research On Driver’s Gaze Direction Estimation Method

Driver’s gaze direction is a key indicator of driver attention evaluation.Its accurate estimation is the basis for distracted driving discrimination and driving behavior understanding.It is of great significance to the improvement of driving safety and the development of intelligent driving systems.Driving is a typical natural application scenario.Due to the influence of natural complex environment,the driver images captured by the optical camera are complex and changeable.Vehicle vibration bumps and the difference in the driving habits exacerbate the uncertainty of imaging quality.In addition,unlike general application scenarios,to ensure driving safety,the driver cannot be allowed to cooperate too much during the gaze estimation process.Therefore,how to establish an accurate,highly robust,and user-friendly gaze direction estimation model based on complex and changeable optical images under the condition of poor driver cooperation is the core problem in the driver’s gaze direction estimation technology.This thesis takes the feature-based gaze direction estimation method as the basic research idea and fully considers various complex factors in real driving scenes.The human eye feature extraction,head pose estimation and gaze direction estimation model construction are studied in detail.1)In the complex imaging light environment,the driver’s eye image captured by the optical camera has high noise and low quality.In addition,the human eye area is small,and the resolution of the human eye image used for feature extraction is very low.In this thesis,automatic sparse coding technology is used to study the method of automatically generating a sparse dictionary of human eye images based on a large number of human eye images and reconstructing human eye images based on the sparse dictionary.By using the sparse coefficients of the human eye image containing high-dimensional shape structure features and the multi-scale spatial pyramid feature construction method,a multi-scale driver eye sparse coding feature extraction method based on global and local dual dictionaries is proposed.Experimental results on public datasets show that the multi-scale human eye sparse coding feature proposed in this thesis can reconstruct the human eye image well,and it is also effective for the human eye image outside sparse dictionary training set.The gaze direction estimation model based on this feature has also achieved high gaze zone classification accuracy in real driving scenes.2)In the driving scene with uneven light intensity,the 3D point cloud data of the driver’s head collected by the binocular camera is prone to partial missing due to insufficient reflection and obstruction of the camera perspective.In this thesis,a stable point cloud construction method for iterative registration of multi-frame point clouds is studied.Based on this method,an accurate estimation of the head pose is achieved by using the transformation matrix between point clouds.At the same time,the method of tracking and predicting the driver’s head posture based on particle filter is also studied.The prediction results of the current state are used to quickly locate the nearest neighbor point cloud template,which improves the efficiency and accuracy of point cloud registration.The experimental results show that the proposed method achieves good estimation accuracy in the three degrees of freedom of the head posture in the real driving scene.In addition,compared with the head pose estimation method based on 2D images,the accuracy of gaze estimation is improved.3)The gaze direction estimation model needs to be recalibrated after the imaging environment changes.For driving safety,it is difficult for the driver to actively cooperate to manually calibrate the gaze direction estimation model.Based on the analysis of the distribution law of the driver’s gaze placement,this thesis studies the implicit extraction method of the calibration point of the gaze direction estimation model.The calibration points extracted based on this method realize the unsupervised online training and automatic calibration of the gaze direction estimation model.In a real driving scene,the proposed method is used to simultaneously train the gaze direction estimation model online based on driver images captured from two different imaging perspectives.Both can achieve a relatively ideal gaze direction estimation accuracy in a short time,and the estimation errors of the two are also relatively small.Finally,this thesis combines the relevant research results of human eye feature extraction,head posture estimation,and automatic calibration of gaze direction estimation models to construct a set of effective driver gaze direction automatic estimation methods in real driving scenes.On this basis,the driver’s gaze direction,a key driver’s attention cue,is applied to driving behavior analysis and a driving behavior analysis and visualization method based on multi-modal data fusion of the driver’s gaze direction and vehicle state is proposed,which verifies the research results of this thesis.