Design And Realization Of Real-time Stereo Vision System Based On FPGA

Stereo vision technology is a very important research part in the field of computer vision,which extracts the depth information in the real scene by matching a pair of images with different view angles.The technology is widely used in unmanned aerial vehicles,unmanned aerial vehicles,virtual reality,human-computer interaction and 3DTV.In the past several decades,many algorithms and design platforms have been proposed to improve t he accuracy and real-time of the depth.Because of the huge computational complexity and high computational complexity of obtaining the depth information of stereoscopic vision,the quality of the depth of information is still a huge challenging work.In this paper,the hardware and software design of real-time binocular stereoscopic vision based on FPGA+ARM architecture are proposed.According to the requirements of the system,the system mainly includes key modules such as high-speed receiving of image sensor video,geometric correction of polar line,Census transformation,semi-global stereo matching algorithm,LRC consistency check and median filter.The whole stereoscopic vision system is implemented on a single Xilinx ZC706 development board.The hardware platform uses the XC7Z045 chip as the processor core.The system is divided into software and hardware functions.PS(ARM)side is mainly to achieve software control and debugging,and PL(FPGA)side of the main implementation of parallel processing of visual hardware acceleration hardware.According to the system supports different resolution output and algorithm configuration mode,the system uses a modular,and parametric design.In the system design process,the Aptina's MT9V034 CMOS camera was adopted for its high-speed LVDS serial transmission interface,which can reduce noise and other interference.The geometric correction of the polar line adopts the 2D regression polynomial equation to simulate the matrix multiplication,which reduces the storage of the two image coordinates and greatly reduces the on-chip hardware logic resources.Pipelined technology is employed during Polynomial calculation which significantly improve the system operating frequency.Due to the complexity of the semi-global stereo matching algorithm and the high memory requirement,this paper describes an effective method to calculate the cost function and simplifies the cost aggregation in four directions,designs a parallel buffer and reduces the memory requirement Median filtering can eliminate the problem of mis-matching points and striping caused by dynamic programming.The experimental results show that the proposed system can meet the real-time video processing requirements,the image resolution of 640 × 480,the maximum parallax search range of 64 pixels,frame rate 60 fps.The matching algorithm module operating frequency can reach 130 MHz.The system can achieve 1280 × 1024@ 72.2 fps real-time video processing.