Research On Zero-shot Learning Methods Based On Generative Adversarial Networks

With the popularity of social media and digital acquisition devices,there are massive amounts of video and image data on the network.By manually collecting and labeling massive data,supervised learning tasks have achieved great success in many aspects,especially in image classification,face recognition,intelligent traffic and so on.However,collecting and marking large amounts of data is very difficult,especially for fine-grained objects and rare objects,which are both time-consuming and expensive.In addition,the most popular deep learning in machine learning is currently driven by big data,that is,the more labeled data,the better the model.However,when the task is more complex and the annotations are less,the model is difficult to learn.Collecting and labeling large amounts of data is very difficult.Therefore,when the number of training samples is insufficient,the samples are unlabeled or even zero samples,how to make full use of the big data constantly generated by the network has become an emerging problem in the field of machine learning and computer vision.In order to solve the above problems,zero-shot learning(ZSL)has become one of the research hotspots in the field of transfer learning and computer vision.The purpose of ZSL is to identify unknown objects.Unknown means that the category of the test samples has never appeared in the training process.ZSL relies on prior knowledge to bridge seen and unseen categories.It hopes that the knowledge learned in the training set categories can be transferred to unseen categories to complete the identification of unknown objects.The idea of ZSL comes from the process of human recognition of objects,and uses advanced semantic attributes shared between different objects as auxiliary knowledge to quickly identify new things.At present,ZSL has been widely studied in the aspects of species recognition,intelligent agent action imitation,face detection and recognition and image scene analysis.At present,most of the ZSL methods study the spatial mapping relationship between semantic attributes and image visual features.Although the models have excellent performance in the training category,but in the test category,the models often suffer from severe performance degradation due to the drift of the mapping domain and the problem of pivot points.In addition,the current mapping-based methods do not perform well on the more challenging generalized zero-shot learning.The recently proposed methods based on synthesized and generated methods often perform better than the mapping methods.In response to the above problems,ZSL based on generative adversarial networks is studied to solve the problems in stages.The main work is as follows:(1)Aiming at the problem of drift in the mapping domain and the problem of pivot points,this paper proposes a novel reverse mapping approach(RRGAN)within a semantic reconstruction and generative adversarial framework by residual network modules for addressing ZSL.Particularly,it synthesizes the perfect visual feature templates through class semantics using neural network with basic residual network modules in generative network and regularizes the visual feature templates by adversarial network and reconstruction network.Moreover,the choice of reverse mapping,the addition of semantic reconstruction modules and the idea of generating confrontation,the combination of the three can solve the problem of mapping domain drift and pivot point at the same time,effectively improve the performance of ZSL.(2)This paper proposes a model with cooperative coupled generative networks(CCGN)based on the idea of RRGAN to solve GZSL problem.Although method(1)can effectively improve the performance on TZSL,but its performance in the latest and more challenging GZSL obviously declines.In order to improve the performance in GZSL,this paper proposes a cooperative coupled generative networks method.Specifically,using different generative networks to combine the generated visual features to finally synthesize more discriminative visual feature centers can effectively improve the performance of GZSL.(3)The above two methods are mainly based on the idea of mapping.This paper also presents a GZSL method based on the fusion of synthesis and generation methods(BFSG).Its purpose is to use the powerful generating ability of the generative adversarial network to generate visual features through semantic attributes and rely on the correlation between semantic attributes to synthesize visual features through semantic correlation.Eventually,samples of unseen classes are synthesized,and ZSL is transformed into a traditional classification problem.This method is superior in both TZSL and GZSL.