Detecting Driver's Use Of Handheld Phone Based On Video Analysis

In recent years,the percentage of traffic accidents caused by driver's distraction is rather high,especially,making a phone call while driving is one of the main reasons that distract the driver's concentration.Although numerous countries and regions have made laws to ban the behavior of using hand-held cell phone while driving,it is difficult to supervise the phone call making behavior in the actual law enforcement process.Therefore,there is an urgent need to replace manual monitoring and judgment with video recording and analysis.Detecting driver's use of mobile phone based on machine vision conforms to the requirements of supervising road traffic security,and has practical significance.In this thesis,we study,design and implement a detection system to detect the driver's use of mobile phone while driving based on machine vision.The detection system collects driver images by installing a front-facing camera on the front windshield in the vehicle.The system mainly includes three sub-modules,which are the region of interest positioning,pattern recognition,motion detection and alarm control.Region of positioning module including face and ear positioni ng.MTCNN is adopted to realize face detection and facial landmarks positioning.The position of the driver is determined through the face area,while the ear area is roughly determined according to the essential proportions of the face info graphic,widely known as "three court and five eyes",to establish the region of interest(ROI).The pattern recognition module is used to classify handheld calls and normal driving images.According to the location of facial feature points,the non-skin color pixels of eyes,nose and lips are removed,and then the single Gaussian model is used as the skin color measurement model to calculate the probability density threshold dynamically and adaptively.The threshold is used to segment the hand image of the ear area,and skin color information is used as the input of feature extraction in the next step.The PCA-HOG feature of hand skin color image in the face and ear region is extracted,and then the feature vector is input into SVM classifier to classify two postures,the phone call posture and non-phone call posture.The motion detection module is used to detect the movement of the lips.Through the lips position determined by the face feature point positioning,calculate the degree of mouth opening by the ratio of width to height of the lips,and then analyze the movement of the lips according to the duration and frequency of the mouth opening,whether the driver is speaking can be judged.Finally,through continuous multiframe images,the classification results of the phone call behavior and the lip movement are counted,and finally the driver's phone call behavior is detected.In this thesis,Python is used to implement the detection system in Linux environment based on Tensorflow and Scikit-learn machine learning framework,and we train and test the driver's call behavior dataset built by ourselves.The final experimental results show that the detection system has good robustness and real-time performance which can effectively detect the driver's hand-held call behavior.