Research On Generative Zero-Shot Learning Based On Multi-Knowledge Fusion

Zero-Shot Learning(ZSL)refers to the image recognition task performed when the label information of the test sample is completely missing.It is a key step and an indispensable way to realize the transformation of artificial intelligence from "perceptual intelligence" to "cognitive intelligence".Generative ZSL is a branch of ZSL methods.Due to its novelty and high ZSL performance,it has been widely studied in recent years.Generative ZSL refers to a type of method to solve ZSL problem by using generative adversarial network(GAN)and its improved version combined with semantic-to-visual space mapping.At present,there are problems such as semantic insufficiency,semantic gap and domain-shift in ZSL,which result in the accuracy and generalization of ZSL models at a low level,which hinders the development of ZSL.Therefore,in order to reasonably and effectively solve these problems in ZSL,this dissertation will propose improved methods from two aspects: semantic feature enhancement and model structure definition,which mainly include the following works:(1)The thesis proposes a knowledge graph based on hierarchical structure classification for ZSL image classification,which provides a biological basis for the relationship between classes in the datasets;(2)The thesis proposes a new ZSL model based on cross-knowledge from the perspective of semantic feature enhancement.This method enables the model to learn more relevant semantic features through the Cross Knowledge Learning (CKL),which significantly enriches the semantic features.At the same time,by introducing taxonomy regularization(TR)item,the model can generate more generalized visual features to increase the intersection with unseen visual features,which significantly alleviates the adverse effects caused by the phenomenon of domain-shift;(3)The thesis proposes a generative ZSL model based on multi-knowledge fusion from the perspective of model structure definition.This method includes a new Multi-Knowledge Fusion Network(MKFNet),which uses semantic features from multiple knowledge for training to learn more relevant semantic information,significantly enriching semantic features to deal with the semantics insufficiency challenge.At the same time,by introducing Knowledge Regularization(KR)item into the MKFNet to increase the intersection with unseen classes to synthesize more generalized visual features,it significantly alleviates the adverse effects caused by the phenomenon of domain-shift;(4)We conducted a large number of experiments on the proposed methods on several major benchmark datasets in the field of ZSL,and proved that the methods proposed in this paper can achieve the current state-of-the-art performance under ZSL tasks.