Study On Zero-Shot Learning Via Deep Generative Models

Zero-shot learning aims to use the labeled instances to train the model and then classifies the instances that belong to a class without labeled instances.However,the training instances and test instances are disjoint.Thus,the semantic description of the classes(e.g.text description or class attribute information)is to establish a connection between the training dataset and the test dataset to make the model effective.The traditional methods are to learn the shared feature subspace.With the rapid development of deep generative models in recent years,how to apply the deep generative models to zero-shot learning and improve the quality of sample generation.In response to this problem,this thesis carries out a systematic study on the zero-shot learning of the deep generative model,mainly including the following research:(1)To solve the problem of single mapping from semantic features to visual features,this thesis proposes A Generative Model for Zero-Shot Learning via Wasserstein Auto-encoder(GWAE),which provides a framework for the following three modules: the generative module that maps the semantic? visual features,the semantic regression that maps visual features ? its corresponding semantic features,and the discriminator to measure the true image features and the fake image features.In this thesis,the generative module and the semantic regression form dual learning,which can guarantee the two-way mapping of visual features and semantic features,and effectively solve the issue of visual features and semantic information domain adaptation.Through extensive tests on four benchmark datasets CUB,FLO,SUN,and AWA2,experiments have proved the effectiveness of the model.(2)Mainstream research work focuses on single semantic information(word2vector or attribute feature)as auxiliary information.The two semantic information of word2 vector and attribute feature correspond to two views of the class,and this thesis considers that word2 vector is a supplement to the attribute feature.This thesis proposes the Multi-view Deep Generative Fusion Network for Zeroshot Learning(GDFN).The GDFN model uses attribute features and word2 vectors to synthesize visual features for unseen class images through a generative model and matches the visual features with semantic features through a regression network.The GDFN model uses the semantic information of two views of word2 vector and attribute features as auxiliary information,which effectively compensates for the unity of semantic space.Experiments on four standard datasets show that better classification results can be achieved.(3)To ensure that the generated samples are highly related to real samples and their semantics,this thesis proposes the Contrastive Prototype Network for Generative Zero-Shot learning(CPNet).The generative network synthesizes visual features by adding semantic features and noise,while the regression network maps the visual features to semantic features.The prototype network learns the meta-representation of classes,and it abstracts the samples with the most semantic information in each class.By comparing the similarity between the virtual prototype and the centroid,the model training can be made more stable,and the target image can be recognized more reliably.Experiments on four standard datasets show the superiority of the model.