Design And Implementation Of FPGA-based Binocular Stereo Vision System

In recent years,stereo vision has been widely used in automatic driving,robot navigation,gesture recognition,three-dimensional reconstruction,intelligent camera,augmented reality and other fields.In general,the stereo matching algorithm has the characteristics of high computational complexity and large amount of data.The general-purpose processor implementation will take more computation time,and it is difficult to meet the real-time requirements of embedded applications,which greatly limits its application in practice.Image processing based on FPGA(Field Programmable Gate Array)can effectively improve the processing speed of the algorithm,but it is more difficult to design the algorithm using hardware description language.In addition,the emergence of HLS(High Level Synthesis)technology has made some complex algorithm designs easy to implement.Based on the current status of this technology,this thesis studies the design and implementation of binocular stereo vision algorithm on FPGA,and completes the design and verification of a stereo vision algorithm.The design of the hardware system is based on the IP core.The main work includes the following aspects:(1)Based on the careful analysis of the Census stereo matching algorithm flow,the Census binocular stereo matching IP core design based on HLS technology is completed.The design of the IP core includes code writing and optimization of the Census transform,Hamming distance calculation,disparity calculation,and disparity optimization.In order to rewrite the algorithm written in high-level language into HLS synthesizable code,the line buffer and window buffer method are used in the design to cache the image data stream and parallelize the strategy to process the pixel.In addition,the median filtering is used to reduce the noise information in the disparity map,and the external interface of the IP core is also designed so that the IP core is connected to other modules of the FPGA.(2)Based on the Xilinx Miz702N FPGA hardware platform,the core module design of the binocular stereo vision algorithm based on Census is completed and a complete binocular stereo vision hardware system is realized.After completing the HLS design of the Census binocular stereo matching module,this thesis builds a complete binocular stereo vision hardware system.The hardware system includes an ARM processor module,a VDMA(Video Direct Memory Access)module,an AXI interconnect module,a VTC(Video Timing Controller)module,a clock setting module,a data format conversion module,and an HDMI display control module in addition to the image processing module.Based on the idea of FPGA hardware and software collaborative design,the software part is responsible for the communication control between the hardware modules of the system.This thesis uses Vivado SDK to complete the writing and debugging of embedded software programs.(3)The performance of the designed stereo vision system has been tested with the public datset Middleburry.Experiments show that the hardware acceleration system of the stereo matching algorithm designed in this paper can process images with frame rate of 24-85FPS(Frames Per Second)under different sizes and disparity search lengths*It can meet the requirements of strong stability,good real-time performance and less hardware resource occupation.This thesis contains 58 figures,20 tables and 38 references.