The Research Of Zero-shot Image Recognition Base On Generative Adversarial Network

The recognition accuracy of the traditional image recognition technology relies on the supervised learning on the dataset with massive labeled samples.However,the labeling of abundant image samples in various types has high requirements for human resources.Therefore,in recent years,researchers are no longer satisfied with applying supervised training methods with labeled samples to learn image recognition models.In this case,the zero-shot learning technology,which transfers the knowledge learned on the target categories containing training samples to the target categories without training samples,has gradually become a research hotspot.The zero-shot learning method based on the generative model applies the generative models such as Generative Adversarial Networks(GAN)and Variational Auto-Encoders(VAE)to learn the conditional probability distribution of visual samples under the constraints of semantic description on the visible class,utilizes the semantic description of the unseen class to generate the samples of the unseen class,and provides new research mentality for zeroshot learning tasks.Therefore,based on the generation of the adversarial network,this thesis starts with the sample quality and network model structure,and has done the following work:(1)Aiming at the hubness problem and domainshift problem in zero-shot learning,from the perspective of the diversity of generated samples and the semantic correlation,this thesis designs Bi-Semantic Reconstructing Generative Adversarial Networks(BSRGAN).In this approach: 1)aiming at the diversity problem of the generated samples,we try to use the loss of the intra-class diversity based on multi-clustering centers as well as the loss of inter-class diversity based on classification model.Through these two losses,GAN in the method is optimized for diversity.2)Aiming at the semantic correlation problem of the generating samples,we design a bi-semantic reconstructing module,which reconstructs the generated image samples into semantic description,and calculate the Euclidean distance from the real semantic descriptions as the semantic reconstruction loss to constrain GAN.3)After the training of GAN is completed,we use the generated samples to train the Softmax classifier so as to solve the problem of the zero-shot learning.When conducting zero-shot recognition,we combine the image samples with the semantic description reconstructed by the bi-semantic reconstruction module to perform the final classification task.The experiments on the four standard datasets(CUB,SUN,AWA2,and APY)have proved the effectiveness of this method.(2)Aiming at the semantic gap problem in zero sample recognition,based on the basic framework of the method in(1),the knowledge graph among different classes of datasets is introduced into zero-shot learning,and a Knowledge Graph based Wavelet Residual Generative Adversarial Network(KGWRGAN)is proposed.In this method: 1)we take graph convolution network as the extracting tool of the semantic features,and combine with the knowledge map to extract features.The extracted semantic features are applied to combine with the noise sampled from the standard distribution,and the adversarial generation network is employed to generate samples of various target categories.2)In the structural design of the neural network,we propose the concept of the wavelet residual connection,and connect neural network in every two layers through wavelet residual connection as a module.For the deep neural network,we adopt the wavelet multilevel decomposition on the initial input to transmit to each module.In the design of generating confrontation network,we utilize a fourlayer wavelet residual network to form generator and discriminator.The zero-shot learning experiment and the generalized zero-shot learning experiment on the Image Net dataset verify the effectiveness of our proposed method.