Multi-layer Weight-Aware Bilinear Pooling And Attention Mechanism For Fine-Grained Image Classification

Image classification technology is currently one of the most popular research directions in the field of machine vision,and has penetrated into all areas of social life.However,people tend to demand more fine-grained object classification in daily life,such as flower species classification,brand classification of car,and automatic recognition of goods,etc.Fine-grained image recognition technology has wide application value in real scenes.The main task of fine-grained image classification is to distinguish different sub-categories in the same categories of objects,compared to the traditional image classification problems,fine-grained image has the characteristics of small differences between different categories and large differences between the same sub-category,therefore how to select and express features effectively is the key to achieve the performance of fine-grained image classification.Based on the existing work,this thesis designs and implements the fine-grained image classification algorithms based on multi-layer weight-aware bilinear pooling and attention mechanism,respectively.The main work of this thesis is as follows:?1?Multi-layer weight-aware bilinear pooling for fine-grained image classification.Bilinear models have been proven to be effective in modeling different semantic parts and capturing the effective feature learning for fine-grained image classification.However,the bilinear model uses only the convolutional features of the last layer for feature fusion,In fact,the convolutional neural network may lose important semantic information in the process of forward propagation,and the mid-level convolutional features will also contain semantic information for fine-grained image classification.The interaction of different convolutional layer features can enhance the learning ability of fine-grained features and improve the classification performance.At the same time,the contributions of different convolutional layers to the final output are different,how to make the deep neural network automatically screen out the useful convolutional layer features remains to be further studied.Therefore,this thesis proposes an effective multi-layer weight-aware bilinear pooling algorithm,which takes the feature interaction of cross-layer object parts through bilinear model as feature representation,and assigns different weights to each convolution layer,and adaptively adjusts the output of the convolutional layer to highlight more discriminating features.The experimental results show that the proposed method has a great improvement in classification performance compared with the existing fine-grained image classification methods.?2?Bilinear attention model for fine-grained image classification.The distinguishing features used for fine-grained image classification are usually located in local areas of the object,and the attention mechanism can obtain the focused image target area,so as to quickly filter out the detailed information of the target and suppress other useless information.On the basis of the existing work,this thesis combines channel attention and spatial attention to enhance the feature representation,and uses bilinear pooling to fuse different attention modules to more effectively perform fine-grained feature learning.In this thesis,the attention module is embedded into the Conv5₃ of the VGG-16 model.Without changing the dimensions of image features,the feature representation is enhanced.Finally,the form of Hadamard inner product is used to replace the bilinear outer product,which reduces the model parameters.Based on the two fine-grained image classification algorithms proposed in this thesis,this thesis conducts fine-grained image classification experiments on the CUB 200-2011 dataset and FGVC-Aircraft dataset.Experimental results verify the effectiveness of the proposed algorithm in this thesis,the model in this thesis has better robustness for weakly supervised fine-grained image classification tasks.