Research On Zero-shot Classification Algorithm Based On Generative Model

In the practical application of image classification,unseen class samples are often not correctly classified because labels are not available.Zero-shot classification uses the semantic knowledge transfer of seen class samples to predict the labels of unseen class samples.In the traditional zero-shot classification problem,the classification task is only for images of unseen classes.In a generalized zero-shot classification problem,the test samples include both seen and unseen class,which is more in line with practical applications.If the traditional zero-shot classification method is used to calculate the cross-modal mapping function between visual and semantic space in the seen class,the domain shift problem will be caused.And it can be partially solved by training the classifier with deep learning and generating models to synthesize unseen class samples.But the generated sample lacks discrimination and will affect the classifier's recognition accuracy for seen samples.The thesis proposes the regression variational self-coding models to solve the domain shift problem,and also proposes the open set recognition and generative adversarial network models to solve the lack of discrimination of the generated samples,which improves the classifier's recognition accuracy of seen class samples.The main work and innovations of this article include:1.Aiming at the problem that domain shift will occur when performing semantic and visual cross-modal mapping based on the mapping model method,a model based on regression variational autoencoder is proposed.Firstly,visual and semantic features belonging to the same category are input into two variational autoencoders,which are combined in the way of alignment and cross-reconstruction.Then the generated pseudovisual features are semantically constrained by the regression network.Finally,the model was used to create the generated samples of the unseen classes,and the two-level cascade softmax classifier was trained with the visible class samples.Based on the result information of the first-layer softmax classifier,the second-layer softmax classifier is optimized,and the classification result of the second-layer classifier is used as the final prediction result.2.Aiming at the problem that the generated samples lack discrimination and that the unseen generated samples will affect the accuracy of the classifier's recognition of seen samples,a model based on open set recognition and generated adversarial networks is proposed.Firstly,to form the antagonistic learning framework with the discriminant network,the regression network and recognition network are added to the conditional variational autoencoder network.The recognition network increases the discrimination of the generated samples by increasing the differences between classes,and the regression network adds semantic constraints to the generated samples.Then an open set recognition algorithm is used to separate the unseen and seen classes in the test sample.Finally,train separate classifiers for the seen and unseen test samples.The problem that the seen class accuracy caused by training is affected.In this thesis,the effectiveness experiments of regression variational autoencoder,open set recognition and generative adversarial network are respectively performed on the data sets of AWA1,AWA2,SUN and CUB.The experimental results show that the two methods proposed in this thesis are more accurate than the popular methods under the evaluation criteria of generalized zero-shot classification.It also proves that the regression-based variational autoencoder model can effectively reduce the problem of domain shift.Based on the open set identification and generation antagonistic network model can reduce the problem that the accuracy of the seen class is affected by the unseen class generated sample.