A Research On Pedestrian Detection Method Based On Fusion Of Visible And Infrared Images

Pedestrian detection is an important part of advanced driver assistance systems.At present,the pedestrian detection research based on visible vision system has achieved initial results,and it has a good detection effect under good illumination,but there are problems of high missed rate and low accuracy in dark light environment.Therefore,in order to adapt to the all-weather complex road environment,the vehicle pedestrian detection system integrating visible and infrared vision sensor theoretically has better accuracy and robustness than the pedestrian detection system that relies on a single visible or infrared vision sensor.In this thesis,aiming at the low detection rate and missed rate caused by the single modal classifier used in the current pedestrian detection method of visible and infrared image,which cannot adapt to all-weather driving scenes.This thesis proposes the application of early fusion and late fusion algorithms of visible and infrared images,and conducts research on pedestrian detection algorithms under complex illumination,the main research work are as follows:Aiming at the problem that researchers usually choose objective assessment metrics and fusion algorithms in the application scenario of visible and infrared image fusion based on experience,this thesis proposes a fusion algorithm selection method based on objective assessment metrics,which can filter out the objective assessment metrics suitable for the current scenario through correlation analysis,consistency analysis and dispersion degree analysis,and obtain the most suitable image fusion algorithm in this scenario by sorting the metrics value of the fusion algorithm.This method has good performance in the early fusion comparison experiment.Due to the problem of reduced detection accuracy caused by the loss of pedestrian features in visible and infrared images directly using the image fusion algorithm in the early fusion pedestrian detection.An early fusion algorithm for increasing the attention mechanism of pedestrian area is proposed,and the salience detection network is used to process the infrared image to obtain the pedestrian saliency area,and then the infrared image that enhances pedestrian features can be obtained after channel fusion with the original infrared image and pedestrian saliency image.Then,the image fusion algorithm selection method in this thesis is used to fuse the enhanced image and the visible image,and finally the detection is performed.Experiments show that this early fusion algorithm has better accuracy than single visible or infrared detection,and improves m AP by 2.7%.How to assign the weight of visible detection and infrared detection is a problem for late fusion pedestrian detection.In this thesis,a late fusion detection framework is proposed,which introduces the illumination perception module to evaluate the illumination intensity of the visible image to obtain the fusion weight of the visible part and infrared part,and then fuses the result of visible and infrared separately according to the weight.In order to further improve the detection result,the dynamic head is used to optimize the head structure.Experiments show that compared with other methods,this method has a certain improvement and reduces the MR by at least 3.6%.