Research On UAV-oriented Visible And Infrared Image Fusion Algorithms And Its Parallel Optimization

Nowadays,improving the perception ability became an important preliminary for further improvements of UAV's intelligence and autonomy.In many cases,it is far from enough for a single sensor to detect target successfully under complex backgrounds,thus multi-sensor image data fusion technology emerges.In this paper,a novel visible and infrared image fusion algorithm to improve UAV 's perception when the target is obscured by smoke or dark nights,is proposed;an optimization of the proposed fusion system based on CPU-GPU heterogeneous platform is presented as well.In summary,the main contents of this paper are listed as followings1.A novel visible light and infrared grayscale images fusion algorithm based on Re-dundant Directional Lifting-based Wavelet(RDL Wavelet)and hybrid saliency de-tection algorithm is proposed.The RDL Wavelet is implemented based on the spa-tial domain lifting steps,and do multi-scale decomposition to input images.Si-multaneous,with the help of Sinc interpolation,the Subpixels of the input image is obtained in order to extract more directional information.In the hybrid saliency de-tection algorithm,the contrast characteristics of the global and local regions of the multi-source images are considered at the same time.Then a weighting coefficient map based on the mixed saliency map is calculated.The weight coefficient map is used for the weighted average between the input image multi-scale decomposition coefficient matrices.The experimental results show that compared to other state-of-the-art fusion algorithms which are based on multi-scale decomposition as well,the proposed fusion system has both better.For objective evaluations,important evaluation indicator QG is improved more than 48.7%,another indicator QY is also improved more than 19.3%.2.The CPU-GPU heterogeneous platform is used to realize the parallel optimization of the developed fusion algorithm.The optimization process mainly includes two aspects:(1)analyze the latency performance of the fusion system,then implement the bottleneck functions on GPU side,and remains are implemented on CPU with multi-thread optimization;(2)Storage optimization of GPU-side program.Storage optimization is to fully exploit the storage hierarchy features of the GPU improv-ing the access efficiency of temporary data,and reduce the storage overhead of the GPU by using In-place and Memory Sharing tricks.Experimental results on the PC platform show that the implementation of the proposed fusion system based on CUDA can achieve a performance improvement of 6.6 times when the input image is 480 × 640,resulting in 34.4 fps of fusion,the storage on GPU reduce by 50%meanwhile.3.Finally,color images fusion algorithm is evaluated on the UAV equipped with em-bedded CPU-GPU platform.The I channel of visible image in HSI color space is used to fused with grayscale infrared image,and the fused image is converted back to RGB color space.The experimental results show that the visible light and in-frared image algorithms improved the UAV's ability to perceive the target in scenes where the target is weak visible under the smoke or the night.At the same time,compared with the CPU implementation within the embedded platform,the real-time performance of the fusion system on GPU is 1.0 fold higher,and the speed is more than 25 fps when input size is 265 × 320.