Zero Shot Latent Space Mapping Method Based On Bimodal Dictionary And Zero Shot Adversarial Generation

Image classification algorithms based on supervised learning require sufficient training samples with labels,but it is impractical to collect sufficient samples for each category.ZeroShot Learning(ZSL)uses modal information shared by categories to transfer discriminant knowledge,which can provide classification basis for new categories.In this dissertation,we focus on visual and semantic bimodal dictionary mapping and zero shot generation method in ZSL image classification.Main works are as follows:Ainming at the problem that visual and semantic features have structural dismatch,which makes the direct mapping difficult.A ZSL image classification method based on local sensitive double dictionary learning is proposed.The visual and semantic association is established by jointly optimizing the latent feature learning and the cross-modal mapping.Since dictionaries and cross-modal mapping functions are learned under image visual and semantic features respectively,the complementary information of visual and semantic is compatible in latent space,which reduces the structural differences.In the double dictionary learning,the local structure information is vital,a local sensitive regularization is proposed to preserve local structure information.In addition,to alleviate the problem of projection domain shift,a domain adaptation based on self-learning is proposed,which uses the structural information of training data and test data to improve the generalization of the model.Experimental results in AWA databases show that,compared with the SJE algorithm,the proposed method is increased by 12.4% in classification accuracy.Aiming at the problem that the classification effect is biased due to the missing data of new classes,a visual feature generation method based on conditional variational autoencoder and adversarial learning is proposed.Firstly,the latent probability distribution of the visual features conditioned on the semantic imformation is modeled by conditional variational autoencoder.Then adversarial training is adapted to enhance the similarity between the generated data and real data.To further improve the discriminative of generated data,a classification loss is adopted to classify the generated data into corresponding class,thus increasing the separability between different classes.Experimental results in AWA database show that compared with the ALE algorithm,the proposed method is increased by 7.7% in classification accuracy.