Study On Algorithm Of Part-based Fine-grained Image Visual Analysis

Existing components of visual analysis have already achieved satisfied results in the task of basic-level visual categorization.However,they failed to get satisfied results in the task of fine-grained visual categorization(FGVC).The difficulties in FGVC includes faint between-class differences,large intra-class variations and highly local features.In this paper,we investigated the technologies of visual analysis in the issue of FGVC,including fine-grained categorization,retrieval and segmentation method.According to the structures of targets in FGVC,we have presented some novel method based on body parts focusing on different aspects of FGVC.Our methods can be dived into three categories according to the way we dealing with the body parts: strongly supervised method using ground-truth part information,semi-supervised method using information from training parts,and unsupervised method which discovers body parts by itself.Specifically,our research includes:(1)We presented a fine-grained categorization method based on supervised part-level sparse coding,in order to reduce the quantization errors caused by feature encoding.The proposed method takes the ground-truth part coordinates and aligns all the body parts in the instances,obtaining highly local features specifying body parts.The pipeline of feature extraction is built on sparse coding.Different from the existing sparse coding method which learns dictionary of visual words,our method directly takes the image patches of body parts as the bases of the dictionary to represent the query images.As it uses the ground-truth part coordinates,it is a strongly supervised method.The part-based sparse coding benefits the discrimination of the features by explicitly maintaining more species correlations from the bases.Otherwise,we also investigated the importance of different body parts on distinguishing different species.Experiments shows that our method is able to get satisfied results under moderate cost of computation on a challenging dataset for FGVC.(2)We presented the semi-supervised and unsupervised part detection and partaware segmentation method for fine-grained categorization,which aims to get accurate foreground segmentations and reduce background clusters and thus benefits FGVC.As for part detection,we designed a semi-supervised method and an unsupervised method respectively.The semi-supervised method is designed for datasets with part coordinates which are used for training a part detection model.We use the trained models to inference part coordinates in unseen images.Our part detection model combines parametric models and non-parametric models,which is able to give accurate predictions of the parts under moderate cost of computation.The unsupervised method is designed for datasets without any part labels.We firstly cluster all the instances in the dataset according to their pose.Then a region-based segmentation method is applied in some seeding images,generating some candidates of parts.After that,these parts are filtered using an average mask obtained by GrabCut segmentations of the instances within a pose cluster,eliminating the candidates from the backgrounds.Finally,these filtered parts are used to train a part detection model in the same way we used for the semi-supervised method.For instance segmentation,our method iterates between updating the segmentation output by GrabCut and the part proposals generated by the detected parts,ensuring the consistency of these two components.Specifically,once a detected part is assigned to be background by GrabCut,then we increase the probability of this part being the foreground in the next segmentation iteration and decrease the probability of it being a part denoted by the part proposals.Experiments demonstrated that the proposed method benefits both the segmentation and classification for FGVC.Moreover,the experiments shown that some basic-level methods can not do well in FGVC tasks.(3)We devised an unsupervised part-based fine-grained categorization method for structural weakness objects.For some objects without explicit configuration of body parts,we use saliency detection to discover set of parts and adapt local feature extraction and pooling which benefits the discrimination of the final features.In our method,we firstly segment the instance via saliency detection based method.Then we compute the saliency map of the foreground obtained in the first step and cluster all the pixels according to their saliency values,dividing the instance into several parts.Although these parts various dramatically in their appearance and poses,they are usually semantic body parts due to the local smoothness of the appearance.After that,we get alignment of these parts and extract local features and compute visual words for these features in each parts respectively.Finally,the local features are pooled using the locally learned visual words for final classification.Experiments show that our saliency-based division scheme is able to highlight delicate local features of a flower and ensuring the expression of these features in the final vectors for classification.(4)We presented a novel task for FGVC which is part-based fine-grained image retrieval.Existing works of FGVC mainly focus on distinguishing different species of finegrained targets such as dogs and planes by investigating the visual similarity of the query instances.However,these methods neither explore the correlations between features across different species,nor investigate the visual similarity between body parts.The proposed task aims to retrieve images based on the similarity between their body parts.Because similar components of the body of an instance hints the functional similarity,which provides cues for study of the evolution of animal body parts and finding cross-species behavioral similarities.We designed the first baseline method for the proposed task,which includes the following novel components: 1)a geometry-constrained part pooling layer in the deep networks,which adjust the receptive field for each part individually.This layer reduces the interferes from other parts and obtains better local features specifying the target parts.2)a two-step retrieval strategy balancing the retrieval efficiency and the cost of computation.3)weights for different parts specifying the discriminative power on distinguishing different species,together with a multi-scale retrieval method based on the proposed baseline method.Otherwise,we also give solutions for the retrieval of structural weakness objects based on body parts.