| The purpose of multispectral image fusion is to generate a more specific and accurate fusion image for scene description,in which the effective information of multispectral image data can be integrated and redundant information can be removed.Multi-spectral image fusion technology can solve the problem of insufficient information content of single spectral image and improve the accuracy of image post-processing,which has great research significance and practical value.At present,multispectral image fusion technology is widely used in military,medical,transportation,security and many other fields.Compared with other multi-spectral image fusion technologies,infrared and visible image fusion has more extensive application scenarios due to the universality and complementarity of source images.In this paper,infrared and visible image fusion technology based on generative adversarial network is studied.The main contents and work are as follows:In order to solve the deficiency of Fusion GAN,such as the lack of multi-scale information,large structural loss and insufficient feature extraction ability,a pyramid GAN based on attention mechanism was designed.The specific improvements are as follows:(1)Use pyramid network to extract multi-scale features;(2)Design attention-based fusion rules;(3)Add structural loss to the loss function.The experimental results show that the improved pyramid GAN method based on attention mechanism can extract multi-scale information of source image well,and achieve good results in most evaluation indexes such as _GQ、_PQ、_YQ and etc.In addition,the improved method not only preserves the infrared target information but also enrich the background details,and the result has a certain generalization.The fusion algorithm Fusion GAN is applied on Hisi platform.Firstly,the fusion platform Hi3559AV100 is determined and the overall framework based on HSI and Fusion GAN is designed by combining the input and computational requirements of fusion algorithm and the unique advantages of Hisi chip.Secondly,the Hisi environment is build and the transplantation of the third party library Opencv are completed,which lays a foundation for the transplantation of the following algorithms.Finally,according to the NNIE development process and the overall architecture of fusion,the model transformation of the algorithm,the generation of wk file,simulation verification and board test are completed.To sum up,the algorithm proposed in this paper has certain improvement in both subjective and objective effects,and the hardware implementation of the algorithm also lays a foundation for the next engineering application. |
