Research On Traget Recognition Based On Feature-Level Fusion Of Visible/Infrared Images

Taking into account the traditional target recognition system based on single-source image,there is no chance to obtain a comprehensive and accurate information description in a complex environment,due to the limitation of the imaging mechanism of the sensor,in addition,the target recognition rate is low.With the continuous improvement of information fusion technology,the space-time coverage of the target description of the system can be expended effectively by the use of multi-source image information,so that the target recognition rate of the system can be enhanced.While for the traditional fusion method,it's well known that it is mainly on the basis of the pixel level.Most of the fusion objects are high-resolution images in large scene,and the fusion effect depends on the image registration accuracy,therefore,it can not be applied to low-resolution target images when it's under local scenes.Feature-level fusion is an information fusion based on the corresponding relation of feature extracted from image.While It has the advantage of ensuring the fusion accuracy,it can calculate relatively small amount of calculation data.At the same time,feature level fusion algorithm can complement the advantages of each source image and feature extraction algorithm,and remove from redundant information.In this paper,due to the recognition of visible / infrared dual source image based on UAV platform,target recognition based on feature level fusion of visible / infrared images is studied in this paper and the main work and innovation of this paper are as follows:(1)A visible / infrared dual-source imaging system based on UAV was built in order to meet the experimental data requirement in this paper,and the multi-attitude and multi-angle dual-source image data sets with multiple targets in myopia field were collected.(2)A multi-class image feature extraction algorithm is made a study,the multi-feature collection cooperation of dual-source image is constructed as feature level fusion object so as to avoid the limitation and sensitivity of single feature description.In addition,the correlation and sensitivity of features in single-source image and dual-source image are analyzed based on the experiments and the direction of feature level fusion is defined for the subsequent feature level fusion.(3)A feature-level fusion method combining mRMR and PCA is proposed so as to optimize the redundancy issue of multi-features in dual-source images.First of all,the multi-feature sets with the characterisics of visible light and infrared single-source images are made a selection and merged respectively with regards to the feature evaluation criterion under the principles of minimal Redundancy Maximal Relevance and then extract the feature subsets that is related to strong target categories from the original festure sets and has low redundancy with other dimensions to be as mRMR fusion feature of the source image.Besides,the mRMR fusion features of visible and infrared dual-source images are transformed and fused again based on Principal Component Analysis,so that the maximized variance and independent principal components can be obtained as the mRMR_PCA fusion features of the target dual-source images.It's showed based on the experimental results that the target recognition rate which is based on mRMR_PCA fusion features of dual-source images has significant improvement than that of single-source images,and its performance is superior to other fusion algorithms.(4)In order to improve the target recognition rate of the system in complex environment with interference,and avoid the limitation of time and feature description brought by multi-feature set construction,a method of feature extraction and fusion recognition based on Convolution Neural Network for dual-source images has been proposed.First of all,the method of the Migration Learning theory has been taken advantage of to train the CNN model and optimize its overall implicit feature learning ability,and obtain the weight and biases parameters of each network layer by the similarly distributed large visible light dataset,on the next step,the visible and infrared images are input into the model,and the final pooling layer output is extracted as the CNN feature of the source image.In the end,the improved mRMR_PCA_CNN fusion features of the target dual-source images are obtained based on the feature level fusion method by the combination of mRMR with PCA.It's indicated based on the experiment that this method not only improves the target recognition rate and stability of the system in complex environments with weak light,noise and occlusion interference,but also improves the system efficiency greatly.