Hardware Architecture For Semi-global Matching In Stereo Vision

As an important branch of computer vision,stereo vision is widely used in many embedded applications including biomedicine,autonomous vehicles,and human com-puter interfaces,etc.Generally,a stereo vision system is composed of four parts:calibration,rectification,stereo matching,and 3D reconstruction.Stereo vision sys-tem computes the depth information of objects by handling stereo image pairs taken by binocular camera from different viewpoints.Because of the existing illumination,noise,specular surfaces,perspective distortions,occlusions,repetitive-texture regions,low-texture regions and so on,stereo matching,which directly decides the accuracy of 3D reconstruction,becomes the most important and difficult step of stereo vision sys-tem.In addition,real-time processing of high definition image is the basic requirement of practical applications.A high accuracy and high throughput full-pipeline hardware architecture for weighted semi-global matching is proposed,with two-row paralleliza-tion and disparity parallelization.The proposed hardware architecture is implemented and verified in FPGA.The major contributions are as follows:(1)A high-accuracy and high-throughput hardware architecture for semi-global matching is proposed,with two-row parallelization,disparity parallelization and five-path parallelization.To achieve full-pipeline implementation of semi-global path cost aggregation,a specific structure based on ping-pong buffer and TDM(Time Division Multiplexing)theory is proposed,which increases the throughput of hardware architec-ture.(2)The software implementations of semi-global matching algorithm aggregate matching costs along 8 or 16 paths.Because some paths are opposite to dataflow,lots of memory resources are consumed to store huge intermediate data.The existing hardware implementations reduce the number of paths to four with disparity accuracy decrease.This thesis proposes a hardware architecture for weighted semi-global matching with five paths aggregation without external memory consumption.The experimental results show a 3.69%improvement in the accuracy.(3)A penalty factor to the intensity gradient,which is enhanced by the Laplace filter,improves the disparity accuracy in discontinuous regions.In addition,SPF is adopted to remove spikes in the disparity map.The sub-pixel estimation adds additional accuracy to the disparity result using corrected equiangular method.(4)This thesis completes the hardware implementation in FPGA.The evaluation results on Middlebury benchmark show that the overall average error rate is 6.03%?The proposed implementation can operate at 156MHz with a throughput of 1280 x 960/197fps considering 64 disparity levels in Altera Stratix V.It can operate at 100MHz with a throughput of 1280 × 960/126fps in Xilinx VC707.However,huge on-chip memory resources are consumed to buffer data due to the using of specific ping-pong buffer.In conclusion,the proposed high-accuracy and high-throughput hardware archi-tecture can be used in embedded applications.