Software And Hardware Co-Design Of Fatigue Driving Detection System Based On PYNQ

With the continuous increase in the number of cars,the problem of traffic safety has become increasingly serious,and traffic accidents have seriously threatened people’s lives and property safety.Research shows that fatigue driving is one of the main causes of traffic accidents.Therefore,it is of practical significance to design a portable fatigue driving detection system with good real-time performance,low power consumption and portable.This thesis adopts the idea of software and hardware co-design,with Xilinx ZYNQ SOC as the core,constructs a non-contact driver fatigue detection system based on FPGA+ARM,which collects driver images,extracts eye features,judges driver fatigue,and HDMI Integrate in an embedded system,and the PYNQ-Z2 development board is used to complete the construction and testing of the system.The system makes full use of the powerful parallel computing power of FPGA and ARM’s characteristics of easy to build an operating system and good control.The main work of this thesis is as follows:In the FPGA part,there are mainly modules such as face detection hardware IP,Video Direct Memory Access(VDMA)and HDMI display system.VDMA is the image data transmission channel between FPGA and DDR,and the HDMI display system is used to complete the high-definition display of the image.Using Vivado HLS high-level comprehensive development tool,using the Adaboost algorithm based on image scaling to be inspected,the face detection hardware IP is designed,which mainly includes image scaling,histogram equalization,integral map calculation,and classifier detection And other modules.The image is scaled using bilinear interpolation,and the integral image value of each pixel in the entire image is obtained according to the integral image incremental algorithm.Considering the FPGA hardware resources,the system adopts the serial-parallel combination to realize the face detection classifier design.The ARM part mainly includes the configuration of the embedded system control environment,video image acquisition,human eye feature extraction,fatigue state judgment,VDMA and face detection hardware IP interactive interface settings,and HDMI data transmission settings.The human eye features are extracted by the method of face alignment.Two methods were used to judge the state of eye opening and closing.The first method uses Eye Aspect Ratio to judge the state of the eyes,and the second method uses the convolutional neural network to judge the state of the eyes.Finally,the fatigue state of the driver is judged by the PERCLOS fatigue judgment criterion.In this thesis,a fatigue driving detection system is implemented on the PYNQ-Z2 development board by using the method of software and hardware co-design.The system test is carried out in the laboratory environment and the car environment.The results show that the system has portability,low power consumption and good real-time performance.