Research On Fatigue Detection Of Ship Drivers Based On Multi-feature Fusion

Nowadays,the circulation rate of inland goods around the country is constantly increasing,the number of ships within the scope of inland rivers in China is also increasing.Although the navigation assistance system of ships is also relatively complete,in the past three years,ship collision accidents caused by fatigue driving of ship drivers on duty have also come up at times.Therefore,it is of great significance to study a fatigue detection method for ship drivers.In commonly used fatigue detection methods,there are issues such as driver resistance,large detection models,high time delay,and high misjudgment rates.Therefore,this article first improves YOLOv5 s to reduce model size and delay in face detection while ensuring accuracy;Secondly,a new fatigue detection model was designed based on multi feature fusion of the eyes and mouth,which reduced the misjudgment rate of fatigue detection;Finally,by designing and developing a fatigue driving monitoring and warning system,human-computer interaction and visualization of detection results are achieved.The main research and work of this thesis are as follows:(1)In view of the low detection accuracy and speed of existing face recognition on Edge device,a method of face and key point detection and location based on YOLOv5 s is proposed.First,the original Mosaic data enhancement method is optimized to make it more suitable for face detection and improve the detection performance;Secondly,using Ghost Bottle Neck to construct CSPNet-G instead of the original CSPNet backbone network makes the detection model more lightweight,replacing the original Focus block with the Stem block,reducing computational complexity,improving the network’s generalization ability,improving the network PAN structure,and improving the detection effect on adult faces.At the same time,based on the difference between the current frame and the previous frame,the detection range is reduced and the ROI is determined,Reduced interference from multiple drivers in the same lens for detection;Finally,Wing Loss is used as the loss function of face key point regression to improve the regression accuracy and convergence speed of the model.(2)To address the issues of high latency and high misjudgment rate in existing fatigue judgment models,Yaw DD,CEW,and NTHU-DDD are integrated to achieve the training of fatigue judgment models for multi feature fusion of eyes and mouth;Secondly,by intercepting and correcting eye and mouth images,the recognition of eye opening and mouth opening and closing is achieved,which reduces the interference of background on eye and mouth state recognition and solves the problem of large head movement during ship driving.Based on the Ghost module,an eye and mouth state classification network EMSD-Net is constructed,and an ECA attention module is added to achieve model lightweight and accelerate recognition of eye and mouth states,A new judgment method has been proposed to distinguish the normal opening and yawning states of the mouth,in order to reduce the misjudgment of normal speaking as yawning and thereby reduce the overall misjudgment rate of fatigue detection;Finally,by integrating eye and mouth features and based on PERCLOS,CCT,and SYT,it is possible to determine whether fatigue is present.(3)A fatigue driving monitoring and warning system for ship drivers has been designed and developed.The system mainly detects input images or video streams,outputs video detection result maps and fatigue feature results,and issues warnings when fatigue status is recognized.To achieve detection portability,deploy the system to the Jetson TX2;To address the issue of video lag after deployment,Tensor RT was used to accelerate the model and ultimately achieve visualization of detection results.