Research On FPGA Design Of Real-time Panoramic Video Mosaic System

With the rapid development of video and image technology,traditional monitoring solutions can not meet people's growing demand.Traditional surveillance system has many disadvantages,such as multi-angle of view,messy interface and so on,which have terrible influence in visual experience of users.Meanwhile,VR technology becomes more and more popular,but the media signal source matched by VR technology is mainly realized by computer modeling,which prevents its development.In order to overcome problems mentioned above,adapting to market demand and improving the competitiveness of surveillance technology,real-time panoramic video mosaic technology emerged as the times require.We can employ this technology to transform traditional multi-view monitoring into single-view panoramic monitoring.simultaneously,this technology can also be used as media supporting for VR devices,which plays an important role in its development.In the market,pure software video mosaic based on CPU architecture is default in the real-time performance.This paper realizes video mosaic by multi-camera video image acquisition and parallel architecture based on FPGA platform to satisfy the real-time requirements.The theory of video mosaic includes image registration,geometric correction and image fusion.Image registration uses SIFT algorithm to find the geometric relationship between images and obtain registration parameters.Because the cameras position is relatively fixed,image registration only needs to be performed once at the beginning of the system opening.In this paper,the video mosaic algorithm is decomposed and implemented.The PC of the host computer completes image registration.The registration parameters are sent to the hardware platform of the FPGA through the serial port,and real-time operations such as image acquisition,geometric correction and image fusion are completed.This paper focuses on multi-camera panoramic video mosaic technology,andintroduces a real-time panoramic mosaic system architecture controlled by PC and hardware platform.Firstly,the research background and current situation of panoramic image mosaic as well as the basic theory of real-time image mosaic are briefly introduced,and the MATLAB algorithm simulation is carried out before the implementation of the FPGA.Then,according to the analysis of video mosaic cases on the market and the implementation of the algorithm,a design scheme of real-time panoramic video mosaic system base on FPGA is proposed,and the functions of each part of the system are briefly described.Meanwhile,based on Quartus software and Modelsim software,the functional design and simulation of each module are completed with hardware description language,including image acquisition and RGB format conversion,SDRAM image storage,SSRAM geometric correction,image weighted fusion and VGA video display.Finally,the hardware verification of the real-time splicing of two video signals is carried out by using INTEL's FPGA chip(Cyclone IV EP4CE115F29C7),including board-level debugging,timing analysis and logic optimization.The system debugging is completed by function simulation of modelsim,debugging of FPGA board level,timing analysis and logic optimization.The results show that the design can output panoramic video with 1280 *768@60Hz resolution.The occupied FPGA resources include logic resources LE6088,memory unit 219520 bits,9 bits embedded multiplier 34,PLL phase locked loop 2.This scheme can achieve better panoramic video mosaic effect,satisfy video the requisition of real-time,and hardware systems take up less resources,which has certain popularization and application value.