Research On Attribute Based Zero-Shot Learning Approaches

With the rapid development of machine learning technologies,especially the rise of deep neural networks,visual object recognition has made tremendous progress in recent years.These recognition systems even outperform humans when provided with a massive amount of labeled data.However,it is expensive to collect sufficient labeled samples for all the natural objects,especially for the new concepts and many subordinate categories.Therefore,how to achieve an acceptable recognition performance for objects with limited or even no training samples is a challenging but practical problem.Inspired by the human cognition system that can identify new objects when provided with a description in advance,zero-shot learning(ZSL)has been proposed to recognize unseen objects with no training samples.Since test classes and training classes are disjoint in ZSL,attributes are adopted to transfer knowledge from seen training data to unseen test data.In this thesis,we research the attribute-based zero-shot learning and propose four methods to improve the attribute representation and current zero-shot learning models.The main contributions of this thesis are summarized as follows:(1)In most of the existing attribute-based research,class-specific attributes(CSA),which are class-level annotations,are usually adopted due to its low annotation cost for each class instead of each individual image.However,class-specific attributes are usually noisy because of annotation errors and the diversity of individual images.Therefore,it is desirable to obtain image-specific attributes(ISA),which are image-level annotations,from the original class-specific attributes.To solve this problem,we propose to learn image-specific attributes by graph-based attribute propagation.Considering the intrinsic property of hyperbolic geometry that its distance expands exponentially,a hyperbolic neighborhood graph(HNG)is constructed to characterize the relationship between samples.Based on HNG,we define neighborhood consistency for each sample to identify inconsistent samples.Subsequently,inconsistent samples are refined based on their neighbors in HNG.Extensive experiments on ZSL benchmark datasets demonstrate the significant superiority of the learned image-specific attributes over the original class-specific attributes in the zero-shot object classification task.(2)The generalization performance of ZSL is governed by the attributes,which transfer semantic information from seen classes to unseen classes.To take full advantage of the knowledge transferred by attributes,we introduce the notion of complementary attributes(CA),as a supplement to the original attributes,to enhance the semantic representation ability.Theoretical analyses demonstrate that the complementary attribute can improve the probably approximate correct(PAC)-style generalization bound of the original ZSL model.Since the proposed CA focuses on enhancing the semantic representation,it can be easily applied to any existing attribute-based ZSL methods,including the label-embedding strategy based ZSL(LEZSL)and the probability-prediction strategy based ZSL(PPZSL).In PPZSL,there is a strong assumption that all the attributes are independent of each other,which is arguably unrealistic in practice.To solve this problem,a novel rank aggregation framework is proposed to circumvent the assumption.Extensive experiments on ZSL benchmark datasets demonstrate that the proposed complementary attributes and rank aggregation can significantly and robustly improve existing ZSL methods and achieve state-of-the-art performance.(3)Zero-shot learning aims to recognize unseen objects given some other seen objects,by sharing information of attributes between different objects.However,attributes with poor predictability or poor discriminability may have a negative impact on the ZSL system performance.In order to eliminate the inferior attributes,we present an iterative attribute selection(IAS)model to select the key attributes for ZSL.Since training classes and test classes are disjoint in ZSL,we design an attribute-based generative model to generate unseen synthesized samples to mimic the unseen test data.Since the distribution of the synthesized data is similar to the real unseen test data,the key attributes selected by IAS based on the synthesized data can be effectively generalized to the test data.Moreover,we theoretically prove that the IAS strategy can improve the generalization bound of the original ZSL model.Extensive experiments on ZSL benchmark datasets demonstrate that IAS can significantly improve existing ZSL methods and achieve state-of-the-art performance.(4)Based on the unseen synthesized data,zero-shot learning problem can be converted to the traditional supervised learning problem.Recent supervised ZSL methods first generate synthesized samples by a conditional deep generative network and then classify unseen test data by a classification network.Since the generative network and classification network are trained separately,the generative network might be optimal at generating samples but not necessarily at subsequent classification.Based on this consideration,we propose an end-to-end supervised zero-shot learning model in which the generative network and the classification network are trained simultaneously.Furthermore,compared to the real test data,the synthesized data are unreliable because of the domain-shift problem.To solve this problem,we introduce the metalearning strategy to further improve the supervised ZSL model.Extensive experiments on ZSL benchmark datasets demonstrate that the proposed supervised ZSL model based on synthesized data and meta-learning is significantly better than other supervised ZSL models.Finally,the summarizations of advantages and disadvantages and the applicability of the proposed methods are presented.Based on these summarizations,several possible future works are introduced.