Research On Image Fusion And Fine-grained Classification Algorithm Based On Attention Mechanism

Under the background of information age,massive image are widely produced everywhere.How to extract effective information is a hot topic at present.Convolutional neural networks with attention mechanism are popular because of their excellent performance.However,feature fusion in multi-focus image fusion and object-level key region localization in fine-grained classification are still difficult to deal with.Therefore,in this thesis,series of researches based on attention mechanism will be made,and the specific contents are as follows:Firstly,a large-scale high quality image fusion dataset is generated.Because of lacking standardized large-scale fusion data,we have to generate large-scale high-quality image fusion data for model training,validation and testing with the help of image saliency detection.The dataset is composed of 256×256 paired out-of-focus images,full-focus images and label masks,which are suitable for all regression networks and some classification networks.Secondly,attention feature fusion network is proposed,and AblationCAM is used for visualization analysis.For feature fusion,an end-to-end fusion network AFFN10 based on attentional feature fusion is proposed.A series of experiments are conducted on Lytro,cell and infrared-visible datasets by comparing seven indexes.Results show that AFFN10 has the best performance in SSIM,AG and other indexes.Finally,we use CAM and Ablation-CAM to generate heat maps for qualitative analysis of the features learned by network.Heat maps demonstrate that Ablation-CAM is more precise in describing details learned by network.Thirdly,transfer learning is used to fine-tune network and data augmentation based on Grad-CAM++ is made.Due to the small scale and multiple categories of fine-grained classification data,we use transfer learning to fine-tune network based on Eifficientnet-B4.Then we locate key areas by using Grad-CAM++ and carry out two kinds of data augmentation on original image according to heat maps generated by GradCAM++.Finally,a fine-grained classification network CSA-VGG16 based on attention mechanism is designed.We combine channel attention and spatial attention to propose a new CSA module,and embed it into VGG16.We conduct experiments on the actual production dataset of cassava leaf disease for comparative analysis.Experimental results show that the CSAVGG16 designed in this thesis has the best classification accuracy of 0.73.