Research On Pedestrian Detection Algorithm Based On Binocular Vision And It's Implementation With FPGA

Real-time perception and recognition of 3D environment are the key technologies in robotics research.The detection of pedestrians and it's three-dimensional position-sensing technology can help robots to serve human beings better and safer,which has huge research value.Compared to other depth sensors,binocular vision system has the advantages of low price,small size,low power consumption,which is very suitable for mobile robot applications.However,the pedestrian detection algorithm based on binocular vision sys tem is computationally intensive and difficult to meet the real-time requirements of mobile robots.Aiming at the actual demand of mobile robot for three-dimensional position sensing and detection of pedestrians,this paper builds a binocular pedestrian detection system based on Xilinx Zynq-7000 platform.The pedestrian's three-dimensional position sensing and detection technologies mainly include three-dimensional environment information acquisition and pedestrian detection.Correspondingly,this paper mainly studies the implementation of binocular parallax calculation and pedestrian detection algorithm with FPGA.Binocular parallax computing module obtain the depth of the environment information through the stereo matching algorithm,which is help for pedestrian detection module.The matching information can be used to verify the pedestrian detection results and improve the detection accuracy.The depth information can use for reducing the pedestrian search area,multi-object segmentation,3D pedestrian positioning,which is important for constructing a more powerful pedestrian detection system.Binocular parallax computing module includes: video synchronization acquisition,polar correction,Census eigenvector extraction,semi-global stereo matching and parallax optimization sub-modules.For stereo matching algorithm,there is a large computational burden in searching.This paper reduces the search area from two dimensions to one dimension through the polar correction of binocular vision system.Because there is a problem that data flow and calculation flow direction are opposite in 8-direction path aggregation algorithm,we choose 3-direction path aggregation algorithm.For the disparity map obtained by the 3-way path aggregation algorithm exisits the phenomenon of stripes and mismatch.In this paper,the parallax is optimized using left and right consistency checking and median filtering algorithm.Pedestrian detection module selects the Histogram of Oriented Gradient as the pedestrian feature and uses linear support vector machine to classify pedestrians.The eigenvectors needed for classification are extracted by FPGA in real time and the classification coefficients and classification thresholds are trained by the PC using the libsvm toolbox.Aiming at the step of the gradient's normalization is too long,we embed the SVM classifier into the eigenvector calculation module.The experiments show that this pedestrian detection system based on binocular vision built in this paper with a good performance.It can meet the real-time requirements of mobile robots and keep a high accuracy in the mean time.The binocular parallax computing module can work normally in the system with 60 fps and the pedestrian detection module's maximum rate is 314 fps.In this paper,the binocular parallax algorithm and pedestrian detection algorithm implemented in FPGA can be used as a visual acceleration module,which is helpful for the robot to perceive the environment.