Research On Stitching Algorithms Of Stereo Images

Based on the principle of binocular stereo vision of human eyes,stereo images display the depth information of images and give people a more real and vivid experience,which can be applied to 3D movies,VR and other related scenarios.The light field camera has a built-in microlens array which can simultaneously record multiple viewpoints of the scene and directly generate stereo images.Due to the fact that the limitations of the hardware manufacturing technology of the light field camera,the field of view of the light field camera is narrowed which impairs the using effect of the generated stereo images.Therefore,stitching algorithm is needed to generate images with a wider field of view.The stereo image stitching algorithm of the conventional camera is complicated in operation and easy to generate vertical parallax,and the use of the light field camera is simpler and more accurate.However,the light field camera just came out and is different from the information recorded by the traditional camera.The traditional image stitching technology can't be fully applied to the light field image.This paper is written to study the stitching algorithm based on the light field image data.When many stereo images are displayed on screens of different sizes,the visual effect is unsatisfactory due to the inappropriate parallax range and the convergence angle between the left and right viewpoints.This paper introduces the methodology to generate the stereo image by using the light field image pair,and the generated stereo image adjusts the display effect of the stereo image according to the convergence angle between the left and right viewpoints,thereby effectively solving the above problem.This paper mainly includes two aspects of work:Firstly,the existing stitching algorithm of the existing light field image is mainly realized by the method of solving the global projection transformation matrix among images.When the target scene is complex or the view transformation between different light field images is large,existing methods would be lack of efficiency.Considering a 3D scene can be approximately viewed as multiple spatial planes(or curved surface),we propose a novel method based on plane fitting and segmentation strategy for light field image stitching.In our method,pixels of the target scene are clustered as multiple slanted planes according to their depth values.A Markov random framework(MRF)based optimization is used to refine the initial rough segmentation.Then,the light rays appearing in different light fields are aligned based on the optical flow information,and the perspective transformation matrix for each slanted plane is calculated for our light field image stitching.Finally,the overlapping regions are merged using the minimum stitching and Poisson optimization method to generate a light field image of a large field of view.Extensive experiment conducted on several sets of real scene light field images demonstrate the effectiveness of our method.Secondly,we propose a novel method for generating stereoscopic images from a light field image pair with flexible control over the convergence of the virtual stereo cameras.We have developed a light field image-capturing prototype that consists of two horizontally arranged light field cameras(i.e.,with their optical axes being parallel to each other).Wecalculate the disparity map of the light field image,and perform perspective transformation based on the parallax information to adjust the convergence angle between the stereo images.When using our proposed device for image/video capture,stereo photographers can concentrate on how to capture the desired visual experience without being frequently disturbed by having to manipulate the stereo camera parameters,i.e.,the convergence angle of a stereo camera.During postprocessing,our method estimates accurate disparity maps for the light field image pair and then generates the target stereoscopic images that satisfy the desired stereo camera convergence requirements by adopting a novel view synthesis method for light field images.Our method can be used for the production of related content such as 3D movies and virtual display.And our method can enhance the viewing experience,reduce the production cost and has broader market expectations.