Free-camera Based Image Mosaic For Multi-planar Scene

Image mosaic is a kind of technique that can synthesize a number oflow-resolution images with overlapped regions from the same scene into ahigh-resolution image. The advantage of this technique can meet the requirementof the observation for the larger image scene. Currently, image mosaic has beenwidely applied in virtual reality, medical image processing, remote sensing andmilitary applications, and it also has become a hot research topic in academicfields, such as computer vision, image processing and computer graphics.In the constraint conditions, the image matching and stitching are moreaccurate and robust. However, no matter in the indoor or outdoor scenes, and thenatural or artificial environment, there are abundant multiple planar structures,rather than one simple planar structure. Therefore, regarding the scene of multipleplanar structures with depth variation, the feature points in the images capturedfrom the scene with different viewing angles will not satisfied with therelationship of the perspective projection transform. It can not achieve the correctperspective and occlusion relationships, so that the following registration will fail.According to the limitation of the existing mosaic algorithms, study of multipleplanar scene image mosaic technology to carry out the view of freedom in thispaper, to reduce the photographer and shooting conditions, the splicing has strongpracticability and flexibility.The main innovation of this article are as follows:(1) We propose a guided sampling based planar structure detection methodusing multiple homographic matrices for image registration. In the proposedmethod, the sorted residual error information is used to guide the sampling ofmatching feature points, and the similar matching points are added to the point setfor sampling. Thus, the inlier probability of sampling point set in our method ishigh, and the parameter model of the optimal homographic matrix can be obtained efficiently.(2) We propose a new method to detect multi-planar regions for imageregistration based on hierarchical clustering for shared nearest neighbor (SNN).The proposed method uses the homographic matrix as the constraint model andestablishes the similar conceptual space. The feature point sets of severalapproximate planes can be obtained after hierarchical clustering for the sparsefeature points. The proposed method doesn’t need to pre-determine the number ofplanar regions for detection. Our method can detect and alignment the planarregions accurately and realize the optimal estimation of multi-model parameters(3) We propose a new method of optimal structure seam selection for imagestitching. For the single planar scence, through analyzing the illuminationinconsistency for exposure time diference and lens vignetting, we adopt thecurvanture value as the matching feature in accumulated error calculation foroptimal seam, which can achieve better robustness. For the multiple planar scence,based on the results of multiple planar image registration, we search for the bestseam which not only meets the minimal structure differences, but also meets therequirement that it should pass through minimum planes. The method can choosethe best projection matrix from the matrixes corresponding to different planesautomatically to complete the image stitching. This method can reduce thestructure errors in multi-planar stitching caused by traditional stitching method.