Hardware-Friendly And High-Accuracy Image Matching Algorithm For Stereo Vision

Depth information plays a very important role in computer vision tasks such as vehicle automatic driving system,robot navigation and 3D reconstruction.Stereo matching is an economical and convenient technique for obtaining depth information from multiple views.In order to realize the real-time stereo vision system on the platform with limited hardware resources,it is of great significance to design the hardware friendly high-accuracy matching algorithm.In this paper,real-time high-precision stereo matching algorithms are studied,and the main work is as follows:(1)This thesis presents a two-cycle time-sharing and resource-efficient pipelined hardware architecture for a semi-global matching based disparity estimation algorithm.The input images are downsampled to improve system throughput.And,column skipping,cost quantization,and other strategies are adopted to reduce hardware resource consumption.To improve the algorithm accuracy,we propose a weighted path cost aggregation.The performance of the presented architecture was evaluated based on the zynq-7 FPGA.The throughput of the system could reach 1280×960@116hz at 200MHz at the 128 dispartiy levels,and the MDE/s was 18245.(2)Based on a convolutional neural network,an end-to-end stereo matching network,FPN(Feature Pyramid Network)-Net,is designed.The ResNet is adopted to extract features,and the feature pyramid network is used for multi-scale information fusion.An hourglass stack network with encoder-decoder architecture is designed to learn more context information in the cost aggregation stage.The average mismatch rate of disparity maps on the KITTI2015 test set was 1.93%,ranking 7th among existing open algorithms.(3)To make FPN-Net more hardware-friendly,we use several model compression methods to reduce network model parameters and the number of floating-point operations to reduce the huge computing and storage requirements.It mainly includes network layer compression of feature extraction module and compact network design of a 3D convolutional network.The parameters were reduced by 75.9%,the FLOP was reduced by 58.9%,and the mismatch rate on the KITTI2015 test set was only 0.42%lower.The fixed-point strategy is further proposed to reduce the storage capacity of the network model by 74%.Experimental results show that the hardware architecture based on a semi-global matching algorithm achieves a proper balance between resources and precision.Compared with other end-to-end networks,FPN-Net based on CNN can obtain a more accurate disparity map by strengthening the learning of context information through feature pyramid network and hourglass network in similar runtime.For FPN-Net,the model compression methods can significantly reduce redundant parameters and floating-point operation times,and the fixed-point strategy can reduce the storage requirements of the model.