Design And Implementation Of Binocular Vision System Based On FPGA

The stereo vision system is able to obtain the depth information of the real scene,realize the three-dimensional reconstruction of some specific scene,and then complete the task of visual surveying and mapping,visual navigation,visual obstacle avoidance and path planning.So it is widely used in some embedded systems such as drones,robots and unmanned driving.Stereo matching is a key part of binocular stereo vision,the matching accuracy and matching speed are of vital importance.However,the stereo matching algorithm is always complicated,it is difficult to meet the real-time and precision requirements at the same time.Due to the high parallelism,flexibility and low power consumption of FPGA(Field Programmable Gate Array),many vision algorithms used in embedded systems are suitable for implementation on FPGA platforms.Xilinx's Zynq has the FPGA+ARM architecture,which combines the parallel advantages of FPGA and the powerful functions of CPU cores.Therefore,the realization of vision system algorithms using the Zynq platform has broad application prospects.In this paper,the binocular vision system is designed and implemented based on the Zynq platform.In this paper,the binocular camera model is analyzed in detail,and the binocular camera is calibrated to obtain the internal and external parameter matrix and distortion coefficients of the camera.The epipolar correction is completed offline,so that the coordinate mapping relationship before and after the correction is obtained.Then the principle of stereo matching is analyzed,which can be divided into the following four steps: matching cost computation,cost aggregation,disparity computation and disparity optimization.Census Transform and Hamming distance are used to calculate the initial matching cost,multi-directional scanline optimization algorithm is used to achieve cost aggregation,Winner-Take-All algorithm is used to obtain disparity,and median filtering is used to optimize disparity.In this paper,the hardware logic part of the binocular vision system is designed in the Vivado development environment,and then the hardware project is imported into the SDK to performs drive configuration and register configuration for the modules used in the hardware project,the captured left and right camera images are simultaneously written to the SD card,thereby realizing image acquisition,image transmission,image access and display system,providing a verification platform for stereo vision IP(Intellectual Property)design.Combined with the calibration parameters and the mapping relationship before and after epipolar correction,the algorithms such as camera distortion correction,stereo rectification,stereo matching,and disparity optimization are implemented in the HLS development environment.The optimization directives are used to complete the algorithm acceleration,and they are packaged into the stereo vision IP core.Using the stereo vision IP core in the hardware platform designed by Vivado,the binocular vision system of binocular image acquisition,image processing,image access and display is finally realized.Finally,the binocular vision system designed in this paper is evaluated experimentally.The experimental results show that the binocular vision system designed in this paper has high real-time performance,obvious hardware acceleration effect,and low overall system power consumption.It is suitable for embedded vision application systems that require high real-time performance and power consumption.