Research On Multiscale,Occlusion-robust And All-weather Pedestrian Detection

Pedestrian detection,which aims to identify and locate pedestrians in input images or videos,is an important issue in computer vision technology and the basis for many tasks such as scene understanding,image retrieval and event detection.With the rapid development of deep learning and processing capabilities of computers,great progress has been witnessed in pedestrian detection during the past ten years.However,in real-world scenarios,there is still a big gap betwwen the performance of pedestrian detection algorithms and the results given by human eyes.Therefore,pedestrian detection has always been a research hotspot in both academia and industry,which has great value in theoretical research as well as in practice.Among the factors affecting the performance of pedestrian detection,scale vari-ation,occlusion and illumination variation are three key challenges.Scale variation means that in input images the scales of pedestrians vary as the distances between pedestrians and the camera are different;Occlusion means some parts of pedestrians are occluded by other objects,thus causing the integrity of pedestrian body structure to be compromised;Illumination variation means that due to the different illumination intensities of scenes,the brightness of different pedestrian regions varies.Focusing on these three challenges,firstly this dissertation carries out a research on handling scale variation and occlusion of pedestrians under common monitoring scenarios.The goal of this research is to learn the feature representation which is robust to scale changes and occlusion.Then,to deal with crowd occlusion formed by pedestrians occluded each other in crowded scenes,which is a special type of occlusion,this dissertation carries out a research on handling crowd occlusion based on context information.The goal of this research is to learn discriminative pedestrian features in crowded scenes.Finally,this dissertation carries out a research on pedestrian detection in multispectral images,and the goal of this research is to improve the performance of pedestrian detection by in-tegrating image information in different modalities under all-weather conditions where the illumination changes greatly.These three parts have carried out in-depth research and discussion on pedestrian detection from different angles,which constitute a rela-tively complete framework for this task.The main research contents and innovation points of this dissertation are as follows:(1).A novel approach for handling scale variation and occlusion in pedestrian detection based on feature reusing and region decomposition is proposed.This ap-proach establishes an augmented convolutional neural network and extracts features of pedestrian candidates from multiple feature layers and background regions by multi-region pooling.At the same time,a box-level multi-scale weak segmentation mech-anism is used to suppress false detection.In this way,this approach explores how to reuse the neural network features more rationally to enhance the ability to handle pedes-trian scale variation.In addition,this approach proposes to employ the visibility scores of pedestrian parts and the classification score of occlusion type to deal with the occlu-sion problem in pedestrian detection.Unlike most existing occlusion-handling methods which design part detectors or optimize feature representation and loss function,this approach innovatively estimates the part visibility of each pedestrian candidate as well as the occlusion type of the candidate as a whole,whose results are fused to obtain a weighted part score to refine the original classification score of each candidate.There-fore,the classification scores of occluded pedestrians are increased and miss detections are reduced.The experimental results show that,compared with existing methods,this approach has excellent ability for handling scale variation and occlusion of pedestrians.(2).A novel approach for handling crowd occlusion based on context infor-mation is proposed.The basic starting point of this approach is to use the rich context information in input images to help detectors to deal with crowd occlusion,which is a special occlusion type.Different from previous methods which directly expand pedes-trian candidate boxes by a fixed value or adaptively to obtain context information,this approach innovatively divides context information into two categories and designs dif-ferent strategies to deal with them,respectively.Specifically,this approach defines the pixel regions around pedestrian candidates as pixel-level context while instance-level context of pedestrians is defined as their surrounding pedestrian instances if they oc-clude each other.For the former,this approach proposes to design a novel pixel-level context embedding module to integrate context information from multiple regions;for the latter,this approach designs a 2-person detector to depict the instance-level con-textual visual characteristics formed by pedestrians gathering together,and designs a novel strategy to fuse the detection results of the 2-person detector and the traditional 1-person detector.The experimental results show that,compared with existing meth-ods,this approach significantly improves the performance of pedestrian detection in crowded scenes.(3).A novel approach for pedestrian detection in multispectral images based on the fusion of deep gated features is proposed.Different from the previous two researches focusing on pedestrian detection on conventional surveillance video images,this approach explores to detect pedestrians under all-weather conditions where the il-lumination intensity changes greatly by integrating the image information of different modalities.The existing methods of pedestrian detection in multispectral images can be divided into two categories,one based on fusion of features and the other based on fusion of detection results.The former suffers from cumbersome model design pro-cedure which requires a large amount of prior knowledge,while the latter suffers from poor performance.In contrast,this approach innovatively designs a gated feature fusion module,which imposes gate mechanism on the features of visible as well as infrared spectral images and integrates them to obtain discriminative feature representations that are insensitive to illumination variation.Thus,the detection performance in poor light-ing conditions is improved.The experimental results prove that,compared with existing methods,this approach has excellent performance in the case of large scene illumination variation.