Research On Image Stitching Algorithm Based On Line Features And Mesh Optimization

In this paper,we introduce a complete process of image stitching and focus on obtaining high-quality stitched image under the presence of parallax.In general,there exist three key steps in image stitching,including registration,alignment with Bundle Adjustment,and postprocess.At the step of registration,we propose a local feature with multiple line descriptors and its unique matching algorithm.Feature extraction and matching are important for registration on image stitching.Previous approaches describe the local feature based on image patch that uses single feature point as the approximate center.But there is no accurate information about orientation or scale in the image patch.On the contrary,line segment possesses it.For this reason,we extract a line descriptor from a model of line segment that links two randomly selected feature points.There forms a mesh topology due to the fact that a line descriptor links two feature points and meanwhile a feature point links multiple line descriptors.But as a price to pay for it,there comes a large number of line descriptors that is bad for matching descriptors.In order to speed up the matching process,we design a unique matching algorithm by exploiting the mesh topology.The result shows that the local feature with multiple line descriptors outperforms other classical features based on image patch on robustness.At the step of alignment with Bundle Adjustment,we propose an alignment algorithm based on mesh optimization.APAP method,who is based on local homographic transformations,has ability of local alignment on conciseness and efficiency,but easy to introduce distortion into stitched image.GSP method can preserve naturalness in stitched image owing to estimation of camera motion and achievement of global similarity transformation.In this paper,we employ local similarity transformations from ARAP method and the combination of two methods mentioned above to take mesh optimization.Mesh optimization not only has universality of alignment method but also accomplishes the target of Bundle Adjustment that simultaneously mapping all original images into unified coordinate.Experimental results show that compared with other methods,the alignment method proposed by this paper not only accurately aligns local detail in stitched image but also preserves structures of original images well.At the step of postprocess,a blending method based on optimal seam-cutting with salient content preservation is proposed.Even though mesh optimization mentioned above can fit image stitching task with little parallax well,it is still hard to solve the problems in two cases that scene with large parallax and scene with moving objects.Generally,a good solution to solve such problems is to find the optimal seam-cutting at the step of postprocess,avoiding directly blending different objects in overlapping region.However,a bad seam often cuts down salient object,producing stitched seam at the result.In order to reduce generation of such bad seam,we employ salient content preservation to weight energy function in seam-cutting method.As a result,the optimal seam-cutting method with salient content preservation efficiently eliminates objects with large parallax and moving objects,meanwhile avoiding cutting salient objects.