Part Localization Of Fine-grained Image And Its Application On Classification Task

Fine-grained image classification aims at distinguishing objects from different subcategories within a general category,which may have marginal visual differences.For example,the images in bird dataset belong to one coarse-grained category but belong to different sub-categories,thus they need to be recognized according to the sub-categories.Compared with traditional image classification task,fine-grained image classification is extremely challenging.The main reason is that fine-grained images present small interclass variance and high intra-class variance,resulting in this task relying on the subtle local differences to distinguish similar sub-categories.Consequently,localizing the accurate parts is a key issue for fine-grained image classification,which can facilitate fine-grained image classification,retrieval and other tasks.Therefore,this thesis mainly focuses on the research of part localization for fine-grained images,and proposes a part localization method under unsupervised setting.And it is applied to fine-grained image classification task.The main work of this paper is as follows:(1)An unsupervised part localization method of fine-grained images based on pattern mining is proposed.Specifically,we first converts the feature maps extracted from a pre-trained CNN model into a set of transactions,and then discover frequent patterns from transaction database by leveraging the pattern mining technique.We observe that those mined patterns typically hold appearance and spatial consistency.Motivated by this observation,we merge the relevant meaningful patterns to generate a support map to localize the possible object and discriminative parts in a fine-grained image.This method is fully unsupervised,and aims to leverage the pattern mining technology to fully mine the semantic and spatial information from a pre-trained CNN model,which reduces the dependence of deep learning on large training data.The localization results on finegrained image datasets such as CUB-200-2011 demonstrate the effectiveness of utilizing pattern mining for unsupervised localization.(2)On the basis of part localization,a fine-grained image classification method based on multi-level features fusion is proposed.This method learns the overall imagelevel features,global object-level features,and local part-level features by building a multi-branch classification network.The global average pooling and global maximum pooling operations are introduced to obtain more statistical information.Finally,the features learned by different branch networks are combined to obtain a more representative and discriminative feature representation,and a new prediction layer is trained to complete the fine-grained image classification task.Compared with other stateof-the-art approaches,the experimental results show that the proposed method achieves competitive performance.Meanwhile,it verifies the effectiveness of multi-level feature fusion mechanism.