Research On Panorama Synthesis Method Based On Multi-camera Calibration

At present,panoramic mosaic technology has gradually become a research hotspot in the field of computer vision because of its wide application prospect and value.Although many researchers have made a lot of achievements in panoramic mosaic and many algorithms have been put into practical applycation,it is found that most panoramic Mosaic algorithms are still only applicable to certain situations,and their universality and real-time performance still need to be improved.When feature points are rare or non-uniform,the existing panoramic mosaic algorithm is not robust enough.Moreover,the real-time performance of the existing panorama synthesis algorithm can not meet the requirements.In view of the above two points,this paper carried out research and proposed a panoramic Mosaic method based on multi-camera calibration and a real-time panoramic Mosaic system based on mapping table and GPU acceleration.The hardware experimental platform was built and verified.The main research work of this paper includes the following aspects:(1)Because the existing panorama synthesis method completely depends on the matching of image feature points,this paper presents a panoramic mosaic method based on multi-camera external parameter calibration.This method takes advantage of the fixed camera position in the panoramic system,obtains accurate stitching parameters by camera external parameters calibration.Compared with the traditional method based on feature point extraction and matching,this method effectively improves the robustness and stability of panoramic mosaic system.In addition,the method of auxiliary calibration by the third camera can effectively solve the problem of stitching parameters between two cameras with low overlapping rate.In this paper,two different camera array combinations are used for experimental testing and verification,and the results show that the method based on multi-camera calibration makes panoramic mosaic system more robust.(2)The registration parameters between multiple cameras need to be obtained by multiplying the external parameter matrix between two cameras.The accumulated errors in this process will make the image splicing misplaced.To solve this problem,this paper uses the nonlinear global optimization method based on the minimum re-projection error to perform global optimization on external parameters.And the internal parameters of image matching estimation are replaced by the calibrated camera parameters to improve the accuracy of registration.Furthermore,in view of the obvious stitching gaps caused by camera imaging inconsistency and parallax,this paper adopts the image post-processing method based on the combination of multi-band blending and optimal seam-cutting to make the panoramic image more natural.(3)Considering the real-time requirement of panoramic video splicing system,this paper proposes a panoramic Mosaic method based on mapping table and GPU acceleration.The method based on mapping table avoids the process of extracting and matching feature points to calculate the image transformation model,and reduces the computational complexity.A DSLR camera is used to rotate the camera every 60 degrees to verify the feasibility of the mapping table as a template.On the basis of the mapping table,the parallel acceleration of GPU was realized,and the panoramic video mosaic system was designed and built.The experimental test reached the speed of 25 frames per second(fps),and the real-time panoramic video mosaic system was almost realized.To sum up,the main research purpose of this paper is to improve the stitching of panoramic system robustness and real-time performance.On the basis of the existing technology,multiple cameras in the panoramic system are calibrated with external parameters using the checkerboard calibration plate,and relatively accurate stitching parameters are obtained to improve the robustness of the splicing system.Moreover,the mapping table is used to avoid repeated operations of feature point extraction and matching,and GPU acceleration is further used to optimize the stitching speed.