The Infrared And Visible Light Video Real-time Fusion System Based On Fpga

Image fusion technology of multi-sensor fuses the same scene into an image, so does the advantages of multiple images. This technology fully utilized better complementarity between the multi-sensors, effectively overcome the single sensor defects and is more reliable, comprehensive and accurate, it has been one of the hottest research technology. The hardware fusion system often uses the architecture of FPGA+DSP, this architecture makes good use of the high efficiency algorithms written of DSP but is likely to cause a lot of waste of resources. In response to these issues, this article uses a single FPGA as the core device, which improves resource utilization and achieves good fusion effect.The research work focuses on the design of the hardware platform and the fusion algorithm modules in FPGA. According to the need of system we design the schematics and PCB board of hardware platform. In software design stage, the paper uses VHDL/Verilog language and configures the registers of ADV7180video decoder chip through the I2C bus; According to the timing difference of the PAL and NTSC, the design can automatically identify these two formats; The paper designs the video decoding module based on the completion of the BT656video formats; The SIFT algorithm is written in Matlab, then we extract feature points and eliminate the error characteristics of the two same scene images and then obtain the transformation matrix of the fusion system through affine transformation model, at last the matrix is written into the FPGA processing module; In order to implement hardware registration, the paper designs bilinear interpolation modules within the FPGA; For the storage of video data, the paper uses SRAM and hung FIFO on both sides of it; In fusion stage, the paper uses the pseudo-color fusion, the weighted average fusion, selects large gray value and Laplacian pyramid method and also analyzes the characteristics of each algorithm. The display of the fusion system uses a generic VGA, so a VGA timing control module needs to be written in FPGA.After timing modify and debug as well as testing indoor and outdoor environment, the fusion system can achieve the real-time fusion of infrared and visible, and supply a good hardware platform for follow-up.