Study On Panoramic Stitching Technology Based On Dual-Fisheye Image

In recent years,panoramic image has been widely used in street view,gym advertising,house view and so on.The traditional panoramic equipment which consists of multiple pinhole cameras is too expensive and large,the single fisheye and circular reflection panoramic equipment has dead angles,while the dual-fisheye panoramic equipment is widely used in immersive panoramic imaging due to its compact size,low price and no dead angle.At the same time,in places such as traffic road and construction sites,panoramic surveillance cameras are also trending to replace some traditional surveillance cameras.The panoramic stitching algorithm of dual-fisheye image is different with the common image,such as the effective area of fisheye image is uncertainty and the complicated stitching process caused by nonlinear characteristics of fisheye cameras imaging model.To address the uncertainty of fisheye image valid region,this thesis studied two methods,one put more weight on the edge information and another put more weight on regional information.Because there is no publicly available data set for the detection of valid regions in fisheye images,this thesis conducted experiments on the self-made data set to verify the two methods.Aiming at the problem that the field of view parameters and rotation parameters of the double fisheye camera are too complex to calibrate,this thesis designed a method to calibrate the dual-fisheye camera parameter and stitch the dual-fisheye image at the same time.The major contributions of our work and the results can be summarized as follows:1.This thesis segment the effective region of fisheye images well by improve the ellipse detector method.The existing edge-based ellipse detection algorithm combines a large number of detected contour fragments with a certain rule to find potential ellipses.This method is sensitive to edge blur problem,and has false detections sometimes caused by not using of the region information of the effective fisheye image.This thesis use multiple scales to solve the slightly blurred edge problem,and use regional information verification to deal with the error detection problem.The experiment shows that this algorithm has high Dice index performance.Most of the time consumption is within 5 seconds.2.The optimization method based on region information is studied to detect the effective region of fisheye images.In this algorithm,the effective area of the fisheye image is modeled as an ellipse,and take the pixel classification error as the region driving term,and add the region driving term,edge driving term and the length regular term together to form the objective function.This algorithm also has a high Dice index.3.A self-calibrated dual-fisheye stitching algorithm is designed.This algorithm optimize the field of view parameter and relative rotation parameter of the dual-fisheye camera using the feature points of ordinary natural scenes.This allows us to directly use a dual-fisheye image with unknown camera parameters for stitching.This thesis use the quality of the stitching effect to evaluate the parameter optimize result.The experiment uses a large number of examples to show the convergence of this method,and the results show that the stitching effect has basically no ghosting,and the matched feature points are very close in the panoramic images.