Design Of Multi-lens Panoramic Imaging System Based On HiSilicon Chip And Front-end Video Stitching

Multi-lens panoramic cameras have played an increasingly important role in the fields of urban security,virtual reality,and remote conferences.The key lies in the implementation of video stitching processing technology.At present,the multi-channel video stitching process in the multi-lens panoramic imaging system often needs to be completed in a PC or graphics workstation that is bulky,high in power consumption,and poor in mobility.Therefore,the development of a multi-lens panoramic imaging system based on an embedded platform with good real-time performance has very important practical significance and economic value.The multi-lens panoramic imaging system involves a lot of logic control and data processing.If an ordinary ARM architecture chip is used as the control and calculation unit of the imaging system,it is difficult to make the system's panoramic imaging process meet the real-time requirements.In view of the above reasons,this article designed a multi-lens panoramic imaging system based on the Hi3559 a chip based on the ARM+GPU architecture,and completed multi-channel video real-time stitching on this system.The main tasks completed in the research process of this paper are as follows:(1)According to the functional requirements of the multi-lens panoramic imaging system,the overall hardware structure of the system was analyzed and designed.According to the analysis results,the main control chip of the panoramic imaging system Hi3559 a was selected.Design the peripheral circuit modules of the panoramic imaging system for the selected main control chip,mainly including multi-channel camera module,communication interface module,storage module and video display interface module,and finally design according to the designed schematic diagram and the electrical characteristics of the selected chip PCB board of panoramic imaging system;(2)Build a panoramic imaging system software development environment on the designed panoramic imaging system hardware platform,which mainly includes system boot program U-Boot compilation and transplantation,kernel compilation and transplantation,file system production and transplantation,driver installation and imaging system software library installation;(3)In order to ensure the smoothness of video stitching of the panoramic imaging system,a multi-threaded video splicing software is designed on the ARM end of the system,which mainly includes multi-channel video capture threads,video format conversion threads,video splicing threads and panoramic video output display threads.The data exchange between threads is completed using a queue-based data structure;(4)The coordinate mapping relationship between the multi-channel video to be spliced and the panoramic video calculated in the video stitching registration stage is stored in a look-up table that combines the neighboring position information of the panoramic video stitching line.Before video stitching,the look-up table and stitching parameters are read into the memory.During video stitching,the OpenCL parallel programming language is used on the GPU side of Hi3559 a to quickly generate panoramic images using the look-up table and splicing parameters read into the memory.The storage space of the lookup table is exchanged for computing time and the parallel computing power of the GPU is used to solve the high time-consuming problem in the video splicing and registration process.