Research On 3D Panoramic Stitching Based On Gradually-changed Camera Array

The image stitching of large field of view(LFOV)and multi-view 3D reconstruction are two hot topics in the area of computer vision.Generally,it is required that very little area between adjacent images are overlapped in image stitching of LFOV.However,in multi-view 3D reconstruction we need cameras to take several pictures of the same scene simultaneously,which provides enough overlapped areas to get the 3D information of targeted scene.Besides,for research such as military reconnaissance and visual navigation of robots,we usually need to get both large field of view and the depth of the object in observed scene,which is always a difficult task for traditional image stitching or 3D reconstruction methods separately.With the combination of hardware design and software algorithm,this paper systematically solves the problem that image stitching of LFOV and multi-view 3D reconstruction cannot be implemented simultaneously.As a novel design of structure,gradually-changed FOV camera array is proposed in this paper and we also make research on the algorithms of image stitching and 3D reconstruction fitting for our designed structure.The main work and innovation includes:1)The structure design of gradually-changed FOV camera arrayThe design of traditional camera array is aimed at image stitching of LFOV or multiview 3D reconstruction only,causing the limited application of the array.A novel structure of camera array is proposed in this paper,which can realize large-field 3D reconstruction and high-resolution panoramic simultaneously.Multiple cameras with gradually-changed field of view are combined in the array,which are regularly fixed in both horizontal and vertical directions.Overlapped field of view between the adjacent cameras in two directions causes that a same spatial point may be captured repeatedly in different cameras,which provides the precondition of 3D space reconstruction.Meanwhile,the gradually-changed field of view of different cameras makes it possible to generate a panoramic image with much larger field of view.2)Multi-view 3D reconstruction algorithm based on patchesConsidering the extraction of fast feature and construction?filtering and expansion of patches,we propose a novel algorithm of 3D reconstruction.Firstly we extract the feature point of Harris and DOG from the captured image and match them.Then a patch is constructed based on the successfully matched featured points.The filtering and deletion of the outliers are carried according to photometric consistency and visibility consistency.Finally we get dense patches of the targeted scene and the depth information of the object inside the scene.3)Image stitching of LFOV with depth informationGenerally,to complete the stitching,we should calibrate the cameras firstly and take the image rectification and projection transformation.The final panoramic image is generated through the matching of overlapped region?the search of best line fusion and image smoothing.This paper propose the improved stitching algorithm with known depth information.During the solution of the global parameters,we add the depth information and imply the Gauss-Newton iterative methods to optimize the function of error.Finally we can get the accurate global parameters of the array and the panoramic image eventually.