The Study Of Zero-Shot Recognition Based On Graph Convolutional Network And Reversible Generative Model

Zero-Shot Recognition(ZSR)aims to solve the task of recognizing target categories without annotated data in the training set.Inspired by the human learning paradigm,the ZSR method transfers the knowledge learned from the seen category data to the unseen category data by means of the semantic correlation between all categories.Most existing ZSR methods try to project the visual features of images and semantic features of categories to a common embedding space,and then use the nearest neighbor search algorithm to determine the category labels of the images in the common embedding space.Because the training set and the testing set contain different categories,and the manifold structures of the visual space and the semantic space are inconsistent,this type of method often has problems such as projection domain shift,mapping bias,and hubness.In order to alleviate the aforementioned problems,the paper uses the Word Net hierarchy structure,graph convolutional neural network and reversible generation model to study the problem of ZSR,mainly including the following three aspects:(1)With the help of Word Net hierarchical structure and description text contained in Word Net,the paper proposes a word embedding algorithm based on the fusion of category hierarchical structure and multi-label semantic features.Specifically,category hierarchical structure information can improve the semantic directivity and discriminative ability of word embedding,and the fusion of multi-label semantic features can help alleviate the lack of visual relevance of word embedding.(2)The existing ZSR methods which based on graph convolutional neural network has some limitations,such as single model architecture and lack of diversity in category relationship graphs.The paper proposes a graph convolutional neural network based on multi-scale category relationships and densely connected mechanism,referred as to MDGCN,which uses the multi-scale graph convolution operation to integrate different types of category relationship graphs into the graph convolutional neural network,and uses the dense connection technology to improve the model's representation and generalization capabilities.(3)The existing ZSR methods which based on generative model has problems such as lack of diversity in generated data and insufficient model expression ability.The paper introduces a novel reversible generative model for ZSR,referred as to conditional normalizing flow-based generative model(CNFG).The CNFG model directly transforms the real data into hidden variables that follow the normal distribution through several affine coupling transformations,and then generates pseudo data instances through the reverse operation of the model.The CNFG model has simple and efficient model construction,clear objective function,and diverse generated data.