Semi-supervised Generalized Zero-shot Learning Based On Modal Fusion

With the rapid development of computer technology,there are more and more applications of machine learning in life.However,the success heavily relies on the large number of labeled samples.Considering that some rare animal and plant samples are difficult to collect,scholars propose a generalized zero-shot learning problem.It is hoped that the model will be able to classify the samples like humans by learning the visual features of the seen classes and the additional attribute information of all the classes.Considering that the unlabeled samples and labeled samples are both taken from life and have similar distributions,making full use of the unlabeled samples in reality can help solve the generalized zero-shot learning problem.This article proposes a semi-supervised generalized zero-shot learning scheme based on modal fusion.The scheme is composed of latent layer feature extraction model and classifier.In order to make full use of a large number of unlabeled samples,the scheme designs a latent layer feature extraction model that is jointly trained with labeled samples and unlabeled samples.It proposes the concept of visual centroid to help reconstruct the semantic latent layer vector into visual features.Bedises,it proposes the concept of heterogeneous semantic latent vector to increase the distance between the encoding result and the heterogeneous semantic latent layer vector to ensure that the generated latent layer features are diverse while not losing the distinction between classes.In the process of classifier training,virtual adversarial training technology is combined to generate virtual adversarial samples for unlabeled samples,so that a large number of unlabeled samples can also be used in training,which will improve the robustness of the model.The two sets of concepts proposed in this plan provide a way to make better use of unlabeled samples and additional attribute information to assist feature extraction models for intra-modal selflearning,and use labeled samples and semantics to complete information interaction between modalities.The semi-supervised learning design is provided to ensure that the unlabeled samples can also be used well in model training.Finally,a comparative experiment on three benchmark data sets proves that this scheme can better deal with the generalized zero-shot learning problem,even if the labeled samples are scarce.