Research Of Zero-shot Image Classification Based On Attribute Learning

Zero-shot image classification is an active topic in the field of transfer learning.Different from conventional image classifiers, samples belonging to testing classes are not involved in the training of zero-shot image classifier. To realize attributes knowledge transfer from seen to unseen classes in zero-shot image classification, the classification model needs to build, by visual attributes(referred to as attribute), one bridge from low-level features to class labels. Attributes refer to features that can be artificially labeled and observed in an image, and are also high-level description of image content understandable by both machines and human beings. Extensive researches have shown the application of attributes in object detection, image description and zero-shot learning.Firstly, we analyzed state-of-the-art of attributes learning, zero-shot image classification and zero-shot image classification algorithm based on attribute learning.Then, with focus on existing deficiency of binary attributes and relative attributes learning for zero-shot learning, we furthermore proposed the improved model with algorithm. The main contributions and novelty are summarized as follows.(1) The original indirect attribute prediction(IAP) model is investigated, which assumes that all attributes are independent and equally important for learning the zero-shot image classifier. However, practically different attributes contribute unequally to the classifier learning. We therefore proposed a relevance probability-based indirect attribute weighted prediction(RP-IAWP) model. We first build the RP-IAWP model by analyzing the relationships between the attributes and classes of training samples. We then predict the attributes of testing samples by using RP-IAWP. Finally the class labels can be assigned to testing samples according to the predicted attributes.(2) When the dimensionality of the semantic attributes is limited, it is difficult for attribute-based zero-shot image classifiers to distinguish objects with similar attributes. Aiming at the limitation of describing objects with semantic attributes, an improved direct attribute prediction(DAP) model for zero-shot image classifying based on hybrid attribute(HA) is proposed, which is called HA-DAP. At first, we carry out sparse coding on the low-level features to obtain non-semantic attributes that are used to assist existing semantic attributes. Then, we take hybrid attributes including the learned non-semantic attributes and manually specified semanticattributes as mid-layer of the DAP model and use the idea of attribute prediction to train hybrid attribute-based classifier. Finally, according to the predicted hybrid attributes and relationship between the attributes and classes, we can recognize the class label for the testing sample..(3) The zero-shot image classification with traditional relative attributes(RAs)needs to train an attribute ranking function for every attribute without considering the relationship between attributes and classes. Therefore we proposed RA based on shared features for zero-shot image classification. Firstly, in analogy to the multi-task learning, the object classifier and attribute classifier are simultaneously learned, from which a shared sub-space of lower dimensional features is obtained. Then we learned the ranking function per attribute by using these shared features. At last, according to the attribute ranking scores, we can predict the test class-labels by maximum likelihood estimation(MLE). Due to the relationship between objects classes and attributes that the shared features included, ranking function obtained can yield high accuracy in attribute prediction and the following zero-shot image classification.(4) For zero-shot image classification with RA, the traditional method suffers from deficiencies such as unreasonably assumed model distribution, subjectivity when modeling and the poor performance of classifier. We therefore propose a novel zero-shot image classifier called random forest based on relative attribute(RF-RA).First, based on the ordered and un-ordered pairs of images from the seen classes, a ranking support vector machine is used to learn ranking functions for attributes. Then,according to the relative relationship between seen and unseen classes, the RA ranking-score model per attribute for each unseen image is built, where the appropriately seen classes are automatically selected to participate in the modeling process. In the third step, the random forest classifier is trained based on the RA ranking-scores of attributes for all seen and unseen images. Finally, the class labels of testing images can be predicted via the trained random forest. Compared with the traditional RA, our proposed method is superior to several state-of-the-art methods in terms of classification capability for zero-shot learning problems.The above four algorithms are carried out on OSR datasets, Pub Fig datasets,Shoes datasets and AWA animal datasets. The experimental results show the feasibility and validity of the proposed algorithms.