Research And Implementation Of Real Time Video Stitching System Based On Multiple Cameras

Image is one of the most important way for human to obtain information in daily life. With the development of information science, digital and video camera have entered people's lives and become a necessary part. However, as more people demand wider filed of visual scenes, the field of visual scene captured by ordinary cameras are limited. Thus, video stitching is needed for people to obtain panoramic video images.Video stitching is to stitch video images with overlapping regions captured at different angles, from different sensors, into panoramic video images. It is a process of dealing with frames from real-time videos, which is still based on image stitching algorithms. The two most important steps of image stitching are image registration and image fusion, which have a direct bearing on instantaneities. Thus, how to accelerate image registration and fusion while guaranteeing the quality of video stitching is a big issue this paper will deal with.The major studies and innovations of this paper are as follows:(1) An advanced video image registration algorithm is proposed based on SURF features. SURF feature detection algorithm is studied and improved according to the demanding of efficiency: Phase correlation is used to calculate the displacement between video images and estimate the overlapping fields between multiple videos,where SURF feature points are extracted for image registration. It helps improve the efficiency of feature detection. In the image registration stage, the search for points matching SURF feature points is performed only in a corresponding displacement region. By doing this, fewer feature points need to be searched, which improves the matching accuracy and efficiency. At last, the transformation matrix that performs projection transformation for the image is obtained after image registration.(2) In the fusion stage, this paper proposes an advanced multi-resolution fusion algorithm. First, an exposure correction algorithm based on a block by block basis is used to deal with discontinuity between video images due to the exposure problem and to correct the color difference between adjacent images; Then, a optimal stitch line is determined in the overlapping region; Finally, to ensure the transition in the overlapping region is smooth, this paper combines the multi-resolution fusion algorithm, stitching images based on optimal stitch line on each level of resolution and reconstructing them into a panoramic video image.(3) This paper designs and achieves a real time multi-video stitching system.Because of the time-consuming nature, the stitching algorithms cannot be used in real-time application. This paper proceeds from GPU programming models and uses CUDA, provided by NVIDIA Corporation, to achieve acceleration and instantaneities during the projection transformation and fusion stage.