Research On High Performance Panoramic Video Stitching Algorithm In Vehicle

In-vehicle panoramic video system is an important technology for ADAS and the360° "eye" of intelligent networked vehicles.Existing in-vehicle panoramic video stitching algorithms based on domestic low arithmetic So C chips are particularly rare,and the functional extensibility is poor.Therefore,to speed up the process of domesticization of in-vehicle panoramic video system and enhance the extensibility of ADAS functions,the thesis uses Allwinner’s T507 So C chip,which is Allwinner’s mass-produced domestic automotive-grade low-arithmetic chip,as the algorithm verification platform to study the high-performance stitching algorithm.The thesis focuses on the details of the in-car panoramic video stitching algorithm to study the high performance in-car panoramic video system that can run on the low-arithmetic Allwinner’s T507 So C chip.The details of the in-vehicle panoramic video stitching algorithm include distorting correcting of the fisheye camera,automatic calibrating of the external parameters,image merging and real-time video generating.In this study,firstly,the stitching logic of in-vehicle panoramic video is studied,and a framework of a high-performance in-vehicle panoramic video stitching algorithm is designed.Secondly,based on the study of the imaging model of non-distortion camera and the calibration algorithm of internal parameters,the distortion model of fisheye camera and the calibration algorithm of internal parameters and distorting coefficients are studied,and a correcting picture calculation method that can change the field of view of the correcting picture is proposed.Then the perspective projection transformation and camera pose,the theories in image space mapping are compared and analyzed,and the high-performance perspective transformation algorithm and 3D texture mapping algorithm are determined,and the alignment algorithm in the vehicle panoramic image is analyzed.For the automatic calibration algorithm of external parameters,a checkerboard pattern corner point detection algorithm based on quadrilateral contour is proposed,which can solve the problem of not recognizing corner points due to the blurred correction picture and large tessellation grid deformation.An image fusion algorithm based on the area ratio of the image fusion area is also proposed based on the installation position of the fisheye camera.A 3D in-car panoramic image based on a ship-like 3D model is realized.In this study,in order to improve the performance of in-vehicle panoramic image stitching,an algorithm is developed to save the position information of pixels on 2D panoramic image or 3D model in fisheye image,and the weight of fusion area as a LUT in png format.When generating the panoramic video at the end,the panoramic image is directly implemented through the LUT.The high-performance algorithm in the above details is deployed on the Allwinner T507 So C chip using the GLSL language of Open GL ES and the Open CV3 open source library,and the final result is an automatic calibration time of up to 10 seconds,as well as a real-time frame rate of 29 fps for the output of 2D car panoramic and 3D car panoramic images running simultaneously.