Binocular Stereo Vision 3D Sensing System Based On ZYNQ And Its FPGA Implementation

Stereo vision is an important research course in the field of computer vision.This technology is widely used in civil,military and aviation fields.At present,most of the researches on binocular stereo vision systems are based on general-purpose computers,which have large volume and large power consumption.And in mobile fields such as mobile robots,drones,and aerospace,especially where power consumption is sensitive and the working environment is complex,it is not applicable to traditional general-purpose computer platforms.Under such circumstances,it is important to realize binocular stereo vision system and research on embedded devices.This experiment uses Zynq-7020 series boards,which has low power consumption and can be free to carry with strong Scalability.Through the coordinated operation of the PL and PS parts,a smooth disparity map of binocular vision is generated.This experiment uses an unconstrained stereo geometric correction algorithm to correct the left and right image in order to reduce the amount of stereo matching calculation.At the same time,the stereo matching algorithm in this paper using the Zero-normalized cross-correlation method(ZNCC),which compares with traditional matching algorithm such as census algorithm,it greatly improves the matching accuracy.This thesis systematically studies the related algorithms of binocular stereo vision technology and hardware circuit design,in order to achieve the stable output of disparity map of binocular stereo vision.The main work of this design is as follows:(1)Using pure C language programming to achieve epipolar rectification and algorithm of ZNCC.Due to the relatively complicated of unconstrained stereo geometric correction algorithm,it is first verified by MATLAB.After successful verification by MATLAB,this design needs to run on Zynq bare-metal ARM and does not use any Opencv function library.herefore,the pure C language program is used to design the unconstrained three-dimensional geometric correction algorithm and bilinear difference method on the platform of Visual Studio 2013 to get corrected pixel image.Then this paper uses the zero-normalized cross-correlation algorithm(ZNCC)to achieve stereo matching based on the corrected pixel image.For a reference pixel of the left image,this paper take the 9 * 9 partial window around it.In the same line of the right image,the pixel in the same size partial window is used to calculate the similarity degree from left pixels to right pixels.The position where the measure of similarity is the largest is the pixel matching point.Finally,the sub-pixel matching points are found by quadratic polynomial fitting,and the disparity values are obtained from the sub-pixel matching points.(2)Binocular stereo vision system with FPGA + ARM architecture through ZYNQ boardThis design uses the Zynq-7020 board independently developed by the laboratory,and proposes a software and hardware system design scheme of binocular stereo vision based on FPGA + ARM architecture.According to the division of system software and hardware functions,the PL(FPGA)side mainly implements binocular camera acquisition and storage and HDMI display,and the PS(ARM)side mainly implements software control and algorithms of binocular stereo vision.This design binocular camera uses OV5640.the image resolution of camera is 640 * 480,and the image transmission frame rate of camera is 30 fps.AXI DMA is used to achieve high-speed data transmission.SGI software interrupts and DMA interrupts are used to implement ping-pong operation.Finally,a disparity map of 640 * 480 is displayed smoothly on HDMI.