Weakly Supervised Fine-grained Image Recognition Via Adversarial Complementary Attention Mechanism And Hierarchical Bilinear Pooling

With the extensive development of deep learning technology,fine-grained image recognition technology has become a hot spot in the field of computer vision.The methods of fine-grained image recognition can be divided into strong supervision methods that need manual annotation of bounding box or annotation point,unsupervised methods that don't need any annotation and weak supervision methods that only need image level annotation.The advantage of weak supervision is that it can greatly save human and financial resources,and also can achieve good recognition accuracy.Therefore,this paper studies fine-grained image recognition under weak supervision.Research on fine-grained image recognition under weak supervision needs to extract subtle discriminative features of the image.Some common methods are directly using convolutional neural networks or methods based on visual attention mechanism to extract salient features of the image.This method can only extract some main salient discriminative features of image,but not other secondary salient discriminative features.When the main salient discriminative features extracted from the two images are very similar,it is not enough to rely on the main salient discriminative features alone.At this time,the secondary salient discriminative features are very important.We try to use a method based on the adversarial complementary attention mechanism,which can not only extract the main salient discriminative features of the image,but also extract the secondary salient discriminative features of the image.This method mainly consists of two classifiers,in which classifier A is used to learn some special salient discriminative features of an object,classifier B erases these salient discriminative features learned by classifier A,and then learns the next salient feature,that is,the secondary significant discriminative features learned from classifier A.The two classifiers learn iteratively in an antagonistic and complementary way until the network training converges.The experimental results show that the fine-grained image recognition method via adversarial complementary attention mechanism can improve the accuracy of fine-grained image recognition.Research on fine-grained image recognition under weak supervision also needs to fuse salient discriminative features between different network layers.At present,the research of fine-grained image recognition under weak supervision often directly predicts the category after extracting the semantic information of convolutional neural network,but it ignores that there is a certain semantic correlation between the features extracted from different network layers of convolutional neural network.Therefore,this paper attempts to use a method based on adversarial complementary attention mechanism and hierarchical bilinear pooling.Hierarchical bilinear pooling can fuse features between different layers,strengthen the correlation between the image features,and make the significant areas learned by the network more representative,so as to improve the performance of classification network.In this paper,we use the adversarial complementary attention mechanism to extract the high-level discriminative features of the image,and then extract the discriminative features of different layers under the same scale,then bilinear pooling the discriminative features of these different layers,that is,the discriminative features of different layers are fused to enhance the representation ability of the image,so as to improve the ability of fine-grained image recognition under weak supervision.The experimental results show that the fine-grained image recognition method via adversarial complementary attention mechanism and hierarchical bilinear pooling has certain competitiveness.