The Research Of Panoramic Image Stitching Using Robust Features

Recently image stitching has become increasingly common, especially stitching of the panoramic image. It has become the basic of image-based modeling and rendering, and visual traveling technology, as the development of Computer Vision and Visual Reality Technology. And it has been used to the area of art and social science, even become people's hobby. The main steps involved in the generation of a panoramic image, i.e., image set input, image registration, image stitching and image merging.The process of image registration aims to find the translations to align two or more overlapping images. Methods fall into two categories– direct and feature based. Direct methods have the advantage that they use all of the available image data and hence can provide very accurate registration, but they require a close initialization, which is typically provided by user input to approximately align the images, or a fixed image ordering. Feature based registration does not require initialization, but traditional feature matching methods (e.g., correlation of image patches around Harris corners) lack the invariance properties needed to enable reliable matching of arbitrary panoramic image sequence. Our use of invariant features enables reliable matching of panoramic image sequences despite rotation, zoom and illumination change in the input images.The first step in the panoramic stitching algorithm is to extract and match SIFT features between all of the images. SIFT features are located at scale-space maxima/minima of a difference of Gaussian function. At each feature location, a characteristic scale and orientation is established. The invariant descriptor is actually computed by accumulating local gradients in orientation histograms. This allowsedges to shift slightly without altering the descriptor vector, giving some robustness to affine change. After extracting SIFT features, we use NN (Nearest Neighbor) to match features. NN is considered under the condition of the feature's descriptor is unique. NN use the ratio of between nearest neighbor point and second nearest neighbor as the matching criterion. When we search the nearest neighbor we use k-d tree search method, which has a high efficiency of searching.From the feature matching step, we have identified images that have a large number of matched between them. At the next stage the objective is to find all overlapping images. Connected sets of image matches will later become panoramas. At first, we select sets of 4 feature correspondences and compute the homography H between them using the direct linear transformation (DLT) method; Then use the robust estimation procedure RANSAC (random sample consensus) to select a set of inliers that are compatible with a homography between them and estimate H . RANSAC method can estimate image transformation parameters using a minimal set of randomly sampled correspondences, and finds a solution that has the best consensus with data; After computing the homography H , we use the nonlinear method Levenberg-Marquart to update, and get the robust transformation.After image registration, we have found images that match to each other, and get the transformation between them. By connecting them we will get panoramas. But some image edges are still visible in the panoramas, such as vignetting (intensity decrease towards the edge of the image), parallax effects due to unwanted motion of the optical center, mis-registration errors due to mismodelling of the camera, radial distortion and so on. Because of this a good blending strategy is important.In this paper, we use the Multi-Band Blending method raised by P.Burt and E. Adelson. In this procedure, the images to be splined are first decomposed into a set of Laplace band-pass filtered component images; Next, the component images in each spatial frequency band are assembled into a corresponding band-pass mosaic using a weighted average within a transition zone which is proportional in size to the wave lengths represented in the band; At last, reconstruct the image by the Laplace band-pass image, and get the image after blending.In this paper, our use of invariant features enables reliable matching of panoramic image sequences despite rotation, zoom and illumination change in the input images; By viewing image stitching as a multi-image matching relationships between the images, and recognize panoramas in unordered datasets; At last, we generate high-quality results using multi-band blending to render seamless output panoramas.In the future, the further researches mainly include compensation for motion in the camera and scene. Panoramas by stitching often suffer from the motions of the optical center. These could be removed by solving for camera translations and depths in the scene; In addition, in the process of image blending, our multi-band blending strategy works well in many cases, but large motions of objects in the scene cause visible artifacts. Another approach would be to find optimal seam lines based on regions of difference between the images.