Research On Real-time Stereo Vision System Based On FPGA

Stereo vision has been used to measure the depth of object surface based on multi-view images of the same scene captured from different viewpoints, and the 3D scene is reconstructed. Stereo vision systems have been widely applied in intelligent mobile robots, e.g., the unmanned vehicle and lunar rover. The real-time implementation of such system has become a hot topic in the field of stereo vision research, and most real-time systems target at small-resolution stereo pairs (e.g.,320 x 240). This paper introduces a real-time stereo vision system based on Field-Programmable Gate Array (FPGA). The platform realizes stereo matching for 512 x 512 images by a three-step algorithm comprised of geometric rectification, census transform, and correspondence matching.Stereo matching is the most time-and resource-consuming component in stereo vision systems. Block matching algorithms are of low complexity and suitable for implementation with hardware acceleration. Therefore, they have been utilized in most real-time stereo vision systems. In this paper, a non-parametric stereo matching algorithm using Census Transform is proposed, which are highly robust for the cases with noises and luminance differences. Besides, this algorithm can be realized with only comparison, XOR, and accumulation operations, and thus it is suitable for FPGA-based logic circuit implementation and acceleration.The stereo vision system is implemented on a stereo vision platform, which has a Virtex5-LXT-series processor as well as four types of storage controllers, including SRAM, Nand-Flash ROM, SDRAM and FIFO. Stereo images are downloaded via a SERDES interface, while the matching results and corresponding disparity values are uploaded through Ethernet.In the implementation of the stereo vision algorithm, the system also adopts several hardware acceleration approaches, such as the ping-pong structure, window-based data cache, and Box filter. For real-time implementation, we have fully exploited parallel processing and pipe-line technique in chip design, which doubles the processing speed of the system by only imposing less than half of the hardware resources. The consistency checking on the two-view disparity maps has been optimized, which costs few resources but suppress a large amount of mismatched points.From the experimental results and performance analysis, it is demonstrated that the proposed system achieves processing rate of 25 fps for 512 x 512 stereo pairs under 40 MHz clock frequency, fully capable to meet the requirement of real-time processing.