| The problem of fine-grained image classification is to classify the same image into more detailed sub categories,which is more challenging than the common image classification problem.Because the difference between fine-grained image classes is small,and the difference between images in the same category is large,the key to solve this problem is to capture the subtle local differences in the image and learn the most discriminative features in the image.Therefore,this paper studies the fine-grained image feature fusion and image saliency,and proposes a bilinear convolution neural network model based on depth feature fusion and salient region resampling,which is applied in fine-grained image classification task.The specific research work includes the following two points:(1)An end-to-end network model,DFI-BCNN(Depth Feature Interactive Bilinear Convolution Neural Network Model),is constructed to solve the problem that the feature fusion level of bilinear networks is not deep enough.The model can be trained by providing weak supervisory information of the image to complete the task of fine-grained image classification.DFI-BCNN has two improvements on the basis of bilinear networks.First,the features between different convolution layers are fused in depth,and bilinear operations of convolution layers 5-1 and 4-1 on the output features of the last network layer are added to fuse the spatial information obtained from the shallow features with the semantic information learned from the deep features.On the other hand,for the highdimensional parameters brought by bilinear operation,two algorithms,random Mc Laurin and tensor shorthand,are added in this paper.By introducing the second-order polynomial kernel to find the approximate low-dimensional mapping of features,the computational complexity is reduced and the efficiency of the model is improved.Experiments on three datasets show that the DFI-BCNN method improves the classification accuracy,and the classification accuracy continues to improve with the deepening of feature fusion,which proves the effectiveness of this method.(2)In order to make the model focus more on the most important areas in the image and learn more discriminative features,a resampling method based on significant areas is proposed.This method has strong generalization and can be embedded in the classification network.This method first analyzes the heat map of the original image,then resamples the significant areas in the image to obtain the resampled image based on the significant areas.The overall model consists of three parts,namely,the generation of significance maps,non-uniform resampling,and fine-grained network classification.First,the thermal map is obtained by using Grad-CAM method,and then the thermal map is binarized to get the significant area image.Then,the significant area is resampled and transformed to the original image by learning the weights of different areas based on the Gaussian kernel function through the sampler,which makes the significant area occupy a larger proportion of the whole image.Finally,the resampled image is sent to DFI-BCNN completes the fine-grained classification task.Experiments show that our proposed method can learn more detailed image features and achieve higher classification accuracy than other fine-grained classification algorithms. |
