Fisheye Lens Calibration And Panoramic Stitching Based On Embedded

Panoramic vision plays an important role in military,medical,virtual reality,and autonomous driving.Panoramic vision can obtain all the image information in the scene and directly or indirectly deliver it to the user,effectively avoiding missing information.The application in the military field can quickly obtain combat information in the scene,and strike the target efficiently and accurately.Applied in the medical field,it can quickly locate the location of the lesion and reduce patient pain.Applications in the field of virtual reality can build complete scenarios and provide a better user experience.When applied in the field of autonomous driving,it can effectively reduce safety risks.However,panoramic vision has not yet formed a universal system that can be applied in multiple scenarios.This is still worth the efforts of many researchers.First,A panoramic vision system framework was designed for universality,real-time,and secondary development.The system framework includes a regular tetrahedral panoramic image acquisition structure,an embedded development platform,network communication,host computer software,and algorithms.This system uses an embedded platform with V4L2 video framework and a four-channel image acquisition card with TW6865 as the processing core to complete the image acquisition.The embedded system is a Linux operating system constructed from the kernel,uboot,device tree and other related source code.The system integrates various function libraries and network functions required by the panoramic vision system.In addition,an image data transmission scheme is designed for the panoramic vision system framework that can span the local area network.Secondly,on the basis of the existing system framework,the image is acquired through remote network communication and the calibration of the fisheye lens is realized.The calibration algorithm is based on Zhang Zhengyou’s checkerboard calibration method,and a fisheye lens calibration algorithm suitable for the panoramic image system is designed and implemented,which lays the foundation for the realization of panoramic image stitching.Then,A panoramic image mosaic algorithm was proposed based on the existing algorithms.The algorithm is based on Pn P algorithm,corner detection algorithm,iterative nearest point solution and other related algorithms,and is designed based on the positional relationship of the multiple channels.This algorithm makes full use of the placement structure of the lens,and has good adaptability to different lens erection structures in different scenes.In addition,the algorithm effectively solves the problem of stitching failure caused by insufficient feature points,and has higher robustness.Finally,the position of the fisheye lens is determined based on the spatial characteristics of the regular tetrahedron,and a panoramic image acquisition structure based on the regular tetrahedron is designed.This panoramic image acquisition structure can ensure that all image data in the scene can be collected.In this paper,the structure is modeled and the relevant model parameters are calculated for the final experiment.In addition,on the basis of the algorithms proposed in this paper,a panoramic stitching algorithm from model to image is designed to improve the real-time performance of panoramic image stitching.In this paper,a panoramic vision system was designed,a panoramic vision model was established,and a panoramic image stitching algorithm was designs for the vision system and model.This algorithm makes full use of the characteristics of GPU high-speed matrix calculation to ensure the real-time performance of panoramic vision.In addition,the combination of image processing algorithms and computer network technology avoids the geographical limitations of the local area network and enables remote observation of panoramic vision.