Research On Pedestrian Detection Methods Via Fusing Visible And Long-wave Infrared Images

Pedestrian detection is a core technology for building an automated driving system.In the all-weather complex traffic environment,the vehicle-mounted pedestrian detection system integrating visible and long-wave infrared sensors has good robustness and accuracy,and is a pedestrian detection system with great development potential.However,the current visible and long-wave infrared pedestrian detection methods are difficult to meet the requirements of automatic driving in terms of accuracy,real-time performance,and scalability.In response to these problems,this thesis combines the National Natural Science Foundation of China project"Infrared Thermography Signal Feature Extraction and Noise Reduction Theory and Method"(51575486)and the Joint Fund Project "Intelligent Vehicle Fault Diagnosis,Prediction and Fault Tolerant Control Research"(U1664264)to carry out the research on " Visible and Long-wave Infrared Pedestrian Detection Method for Autonomous Driving",which focuses on multi-classifier fusion,features optimization,real-time pedestrian detection,and cross-platform transfer learning.The main research contents are as follows:For the single classifier used in the current visible and long-wave infrared pedestrian detection methods,it is impossible to effectively identify the pedestrian features that are different in the daytime and nighttime illumination environment during the around the clock driving,resulting in a large number of missed detections.The multi-classifier fusion method based on illumination aware mechanism is studied to eliminate the influence of illumination environment on detections and improve pedestrian target detection accuracy.Aiming at the problem that the current pedestrian detection methods can not sufficiently learn the background features of visible and long-wave infrared images collected in the complex driving environment,this thesis studies the background feature optimization method based on the region supervision for visible and long-wave infrared images.By adding region supervision modules to visible and long-wave infrared pedestrian detectors,the background features of visible and long-wave infrared images are optimized.Thus,the pedestrian target detection accuracy is improved.The current pedestrian target detection methods for visible and long-wave infrared images are not able to meet the real-time requirements of the automatic driving system.This thesis explores the visible and long-wave infrared pedestrian area detectors,and optimizes the small-scale pedestrian features by building a multi-layer fusion network.The pedestrian area detectors can be trained and tested with low-resolution images to reduce the computational workload and improve the detection speed,which not only meets the real-time requirements of the automatic driving system,but also further enhances the pedestrian area detection accuracy.Due to the differences in camera parameters,installation position and angle between different visible and long-wave infrared image acquisition platforms,the detection accuracy of pedestrian detectors on cross-platform data will be greatly reduced.In order to significantly improve the detection accuracy of visible and far infrared image pedestrian detectors on cross-platform datasets without costly labeling effort,this thesis focuses on the cross-platform unsupervised transfer learning methods of visible and long-wave infrared pedestrian area detectors.Pesudo labels of pedestrian area and parameters of deep neural networks are optimized together.The label consistency and feature complementarity of visible and long-wave infrared images are used as supervisored information to achieve unsupervised transfer learning of pedestrian area detectors and improve the the detection accuracy on cross-platform data.In this thesis,a vehicle-mounted image acquisition system is built using visible and long-wave infrared cameras to collect image data of the vehicles driving process and establish a cross-platform pedestrian dataset.Through the unsupervised transfer learning of pedestrian area detectors on the established cross-platform pedestrian dataset,the universality of the proposed method is verified.