Research On Several Problems For The Practical Use Of Stereo Vision

As one of the most important part of computer vision, stereo vision has advantages of high reconstruction resolution, relative high adaptability and low power consumption. However, the stereo vision is easily affected by the image noise, lighting condition, scene structure and cameras configuration. It is a great challenge for using stereo vision in many application domains which require high-accuracy and real-time performances. Through decades of development, except for few situation such as Mars rover navigation, it is hard to say success in practical use of stereo vision. Taking the performance requirements of practical application into consideration,, doing research and analysis on specific aspect such as problem modeling, computational method and system design, will benefit the po-tential applications.Based on the research background, this thesis presents a critical study to improve the accuracy, adaptability and real-time performance of the stereo vision. With techniques of sparse reconstruction guidance, stereo vision based on spherical surface imaging model, multi-exposure disparity fusion and dedicated hardware system design, the general dif-ficulties in use of stereo vision with complex scenario, wide field of view, high contrast environment and realtime process condition can be effectively addressed.The main contributions are outlined as follows.1. We proposed an prediction model guided depth reconstruction framework. The global stereo matching algorithms generally outperform the local stereo matching algo-rithms in according to the typical benchmark results. However, the real stereo pairs include more complicated scene structures and the fronto-parallel prior model widely used in the global stereo matching algorithms will not be able to match the slanted and curved surface. For these reasons, we proposed an efficient prediction model generation method based on the sparse reconstructions of matching feature points. The prediction model was used to guide the dense global stereo matching procedure and depth refinement. This method improves the accuracy of reconstruction results.2. We proposed a large filed of view environment modeling method based on stereo vi-sion using spherical imaging model. Conventional stereo approach with perspective model cannot applied for environmental perception in large field of view(FOV). For this reason, we define the stereo vision problem based on spherical model. Stereo vision based on spherical model is theoretically possible to estimate the depth in-formation within any region of the camera's FOV. Before stereo matching, we use latitude-longitude sampling method to convert spherical model into equiangular im-ages with horizontal corresponding epipolar lines. By analyzing the characteristics of the equiangular image pair, we chose the semi-global stereo matching method, taking into account the computational complexity and matching accuracy. One step further, we deign an high dynamic stereo vision method based on the multi-exposure which can obtain the three-dimensional information of the high-contrast scenes.3. For the requirements of mobile system, we designed a hardware system oriented for real-time stereo vision computing. Stereo vision algorithms have high computational complexity. The instruction cycle time delay caused by numerous repetitive oper-ations causes the realtime processing of stereo vision to be difficult when using a conventional computer. In order to solve this problem, we designed a hardware sys-tem uses a single field programmable gate array(FPGA) as core processing unit. For the stereo vision computing needs, we designed the data processing flow on system and control techniques for these complex data stream.4. We proposed a FPGA hardware implementation of high performance stereo vision algorithm. Our algorithm exploits a novel cost aggregation approach called adaptive weights method. Recent research shows that algorithms using adaptive cost aggrega-tion approach greatly improve the quality of disparity map. However, this aggrega-tion scheme is very expensive in terms of computation. With hardware friendly ap- proximation, we demonstrate the feasibility of implementing this expensive compu-tational task on hardware to achieve frame-rate performance. The highly parallelized pipeline structure makes system be capable to handle 640×480 pixels image at over 51 frames per second. Compared with other stereo vision hardware implementations, our implementation is among one of the best performing local algorithms in terms of quality. The conclusions and perspectives are presented in the end of the dissertation.