Class Knowledge To Semantic Fusion Based Zero-Shot Learning For Image Classification

As deep learning develop rapidly,it has applied to image classification tasks by a large number of researchers and research institutions,which makes the performance of traditional image classification tasks been improved unprecedentedly,and its recognition accuracy has surpassed that of human beings.However,the traditional image classification task requires plenty of training images and corresponding labels to build an image classification model with excellent performance.Collecting lots of image samples of specific scenes and categories is a very tedious and time-consuming work,not to mention the labeling of classes labels.There is no doubt that this phenomenon has become an obstacle to the further development of image classification task.In addition,one of the biggest limitations in the traditional image classification task is that the classifier trained for some scenes can't be transferred to other scene recognition tasks,which makes the trained model helpless in recognizing new categories.Therefore,how to achieve accurate classification of unseen images in the absence of training samples and how to make the model equip with the transferability become an emergency problem to be addressed.Fortunately,the emergence of zero-shot learning makes it possible for the lack of training samples to predict the unknown samples.Zero-shot learning aims at the situation that the training set(seen class)and the test set(unseen class)have no intersection on the sample class,which makes it impossible to classify the test set.By learning the mapping relationship between the training image and the corresponding auxiliary information(attributes,text and other knowledge),the model has the ability of knowledge transfer,and then the unseen class can be classified by the auxiliary information of unseen class.At present,the existing zero-shot learning methods can be summarized into three different mapping relationship learning methods,which are semantic to visual space mapping,visual to semantic space mapping and semantic /visual to third-party public space mapping.The mapping space is used as the bridge between the visible class and the unseen class to realize the classification of unseen class samples.As zero-shot learning has been researched in full swing during recent ten years,it has obtained a lot of research from researchers and institutions,and has made significant research progress,which makes the zero-shot image classification methods can still achieve good classification effect in the case of missing training samples.However,zeroshot learning is still a big challenge in ensuring the consistency between semantic features and visual features.Hence,for the purpose of solving the existing problems,this thesis studies zero-shot learning and mainly includes the following work:(1)This thesis puts forward a fusion model based on class knowledge to semantic features.By extracting relevant semantic information from the open knowledge corpus,the model performs cross-knowledge fusion for multiple categories of semantic information with high similarity,which provides rich semantic information for zero-shot learning and effectively alleviates the problem of insufficient semantic information of a single category;(2)This thesis raises a zero-shot learning model of cross semantic to visual feature adversarial generation.The model uses a regularization term which can optimize the semantic to visual space mapping function.The regularization term uses triple loss to limit the samples generated by the generator according to the large inter-class distance and small intra-class distance,which effectively improves the quality of the samples generated by the generator.In addition,this thesis also put the semi-supervised learning combined with zero-shot learning.In this method,the unseen samples are added into the original data set to retrain the semantic to visual feature transformation model,which can effectively enhance the learning ability and recognition ability of the model to the unseen information,thus alleviating the negative effects caused by domain drift;(3)We have done a lot of experiments on several benchmark datasets,and proved that the proposed method can achieve superior performance than the latest methods.