Research On Pedestrian Detection Using Light Filed Imaging

Pedestrian detection is a key problem in computer vision,which plays an important role in automatic driving,scene understanding and event judgment.Pedestrian detection is similar to object detection task,that is,identifying and locating pedestrians in input image.With the rapid development of deep learning and general object detection,pedestrian detection has been greatly developed,but compared with human eyes,its performance still has a lot of potential.The common problems existing in pedestrian detection,such as occluded pedestrian,small scale pedestrian and hard negative samples and so on,still need to be solved.In recent years,many pedestrian detection methods based on stereo vision have emerged in academia to solve the above problems,but these methods will also bring other problems.For example,structured light is an active and non-friendly imaging system that can cause damage to human eyes.The binocular vision system needs accurate calibration,and two sensors are required to be used together in practical use,which causes inconvenience to actual application.As a new type of imaging device,the light field camera can simultaneously record the direction and angle information of light at the same time in one exposure.Therefore,light field imaging has two characteristics:(1)In one imaging,RGB information and depth information of the object can be obtained at the same time;(2)After the exposure is completed,the refocused images can be obtained by corresponding computational imaging theory.The depth information of the light field image can help the detector to enhance the ability to judge the negative sample,and the light field refocused images can provide more abundant feature expression space for the detector.In order to solve the above problems in pedestrian detection research by using light field imaging,this paper first establishes a light field pedestrian dataset and carries out research on pedestrian recognition.Then,for small-scale pedestrians,this paper uses light field refocusing technology to build a multi-focus detection network to improve the feature expression ability of small pedestrians.Finally,for occluded pedestrians,this paper constructs an occluded pedestrian detection network based on light field information to improve pedestrian detection performance in the situation of occlusion.These three parts study and discuss pedestrian recognition and detection from different angles,and form the main research framework of this paper.The main contributions and innovation points of this paper are as follows:First,we construct a light field pedestrian data set with 3500 samples,and divide the dataset into three parts according to the research task.These three parts are suitable for pedestrian recognition,small size pedestrian detection and occluded pedestrian detection.The dataset contains pedestrian samples in a variety of complex scenes,which can truly reflect the various scenes of pedestrians in the real world,and has strong practicability.On the basis of pedestrian recognition dataset,we construct a light field pedestrian recognition network which combine RGB information and depth information of the light field image to solves the problem of fake pedestrian recognition in two-dimensional plane effectively.Second,for small-sized pedestrians,we propose a pedestrian detection method based on light field refocusing images.Firstly,a multi-path detection branch is constructed by using multiple light field refocused images.Secondly,in order to select the optimal detection result from the multi-path detection branch,we propose a cumulative probability selection strategy,which can update the weight of each detection branch through back propagation based on neural network training,and then select the best candidate bounding box in each detection branch.Experimental results show that our method can improve the detection ability of small-sized pedestrians and effectively suppress the detection of negative human-like samples.Third,we propose an occluded pedestrian detection network based on light field information.This network takes multi-focus detection branch constructed by the light field refocused images as the major network.On this basis,we construct a Guided-RPN Network by light field depth information to screen redundant candidate bounding box generated from original region proposal network.Then,we use the cumulative probability fusion strategy to filter the detection results of each refocused branch to get the optimal detection results.Experimental results show that our proposed pedestrian detection framework based on light field information can effectively improve the detection performance of occluded pedestrians.