Research And Implementation On Driver Fatigue Detection Based On Multi-Visual Information Fusion

Fatigue driving is one of the major causes of traffic accident, and driver fatigue detection has become hot spots of Intelligent Transportation Systems (ITS). This paper first summarizes and analyzes the current reaearch of driver fatigue detection that based on computer vision and proposes a new method using multi-visual information fusion. This method first proposed a two-camera co-location and tracking facial approach that effectively improve the accuracy of facial information collection. Then obtain real-time eyes, mouth, head movement and fatigue-related visual feature information through a series of facial feature extraction and tracking algorithm. Finally, we propose an improved Bayesian algorithm to estimate driver's fatigue over integration of visual information.The core study of this paper include: research of dual-camera-based human face location and tracking algorithm and implementation, eyes and mouth features real-time detection and tracking algorithm and implementation, a variety of visual fatigue characteristic information extraction, implementation and improvement of multi-information fusion algorithm for Bayesian.This paper first presents a two-camera-based face location and tracking algorithm. We use two cameras to get real-time video of the driver, including a fixed camera which is used to shoot driver's upper-body and a controllable camera which is used to shoot driver's face. Then we propose a face positioning method that combines skin color segmentation and face validation and a tracking method that bases on CAM Shift and face validation to quickly locate and track the face location from camera A.Then the system sends PTZ control commands through serial port according to the face position in camera A to make the high-speed controllable camera B rotate in real time and follow the driver's head. The results showed that this method can effectively improve the accuracy of facial image collection, access to a large solution of facial image and also could more precisely extract facial fatigue information compare to the usual method. At the same time, we could obtain the driver's head movement information during the processing of camera A video images.In the paper, not only get head movement information, we also propose and improve some real-time detection and tracking algorithms of eyes and mouth's features which collect eyes and mouth's status as the main features of the estimated fatigue from the real-time video of camera B. This paper presents a eye's locating and tracking method based on particle filter. Firstly, we detect eyes based on haar-like features in face region, then initialize particle filter through the detected eyes. In order to improve the algorithm's accuracy and reduce the impact of environmental noise and interference, this paper presents a first-order filtering algorithm to amend the track result and determine whether the eye tracking is lost according to the history face position and eye's position. If it is lost, we will detect eyes with haar-like features and re-initialize particle filter.Mouth locating is according to face distribution. This method is fast and easy to realize. The principle is to locate mouth region according to eye's size and position.This paper selects a variety of visual features as the fatigue characteristics, including Perclos, blinking frequency, yawning frequency and nodding frequency. We also analyze some related visual fatigue calculation method and propose a new method judging the opening and closing state of eyes and mouth. The principle is based on the external rectangle ratio and area of eyes and mouth. Comjpared with the other methods, this method could more accurately represent the state of the eyes and mouth. This method is the foundation method of obtaining Perclos, blinking frequency and yawning frequency. Nodding frequency is obtained by tracking head movements.Finally, we use Bayesian method to integrate the fatigue features that metioned above. As the transition probability of dynamic Bayesian networks is difficult to obtain, we propose a probability updating method to integrate the fatigue features based on Bayesian. And we achieved good results in the indoor environment simulation.