Few Shot Learning Algorithm Based On Semantic Information

With the rapid development of artificial intelligence,deep learning appears in every aspect of daily life.It plays a vital role in computer vision,natural language processing,and multi-agent reinforcement learning.Due to the large number of network parameters,it requires enough labeled data for training to avoid over-fitting problems and obtain generalization ability.However,limited by a series of problems such as high annotation cost and data scarcity,how to use a small amount of labeled data for training has gradually attracted people's attention.The existing few shot learning methods mainly focus on image classification,which aims to learn a classifier to predict the category of an image.Nowadays,there are many works on traditional few shot image classification.In contrast,other areas that are more practical and more difficult are developing slowly.This paper mainly focuses on two areas: zero-shot learning and few-shot incremental learning.The former tries to classify the categories without any labeled data,which can deal with the lack of labeled data due to privacy and other issues.The latter wants to provide the model with incremental learning capabilities to adapt to the continuous arrival of data.In terms of zero-shot learning,the Heterogeneous Graph for Knowledge Transfer method(HGKT)proposed in this paper addresses the inefficient use of data relationship.Our algorithm captures inter-class and intra-class relationship jointly by constructing a heterogeneous structured graph.Based on this graph,HGKT algorithm utilizes graph neural network to learn the reverse mapping from semantic space to visual feature space,which alleviates the hubness problem and transfers konwledge effectively.In terms of few shot incremental learning,Multi-Modal Multi-Model Few-shot Incremental Learning algorithm(2MFIL)are proposed to learn arrived new classes and solve catastrophic forgetting.Specifically,this paper introduces semantic modal information and integrates it with visual feature information to learn more accurate decision boundaries.Moreover,our algorithm uses multiple historical models for knowledge distillation to alleviate catastrophic forgetting effectively.Finally,this paper models few shot incremental learning as bilevel optimization problem,and obtains better performance through the alternate iterative learning.The main contributions of this paper are as follows:1.This paper captures inter-class and intra-class relationship jointly by constructing a heterogeneous structured graph,which can deal with the inefficient use of data relationship and obtain more accurate topology.Moreover,HGKT utilizes Wasserstein metric to extract more presenative node of each class.2.Our approach is the novel inductive GNN-based generalized zero-shot learning method that is agnostic to unseen information during training.Konwledge is transferred from seen classes to new unseen classes based on the learned aggregation and embedding functions.In addition,HGKT algorithm achieves state-of-the-art performance on many public benchmarks.3.In the field of few-shot incremental learning,this paper combines semantic information and visual feature information simultaneously to learn the decision boundary for better performance on arrived classes.In addition,since a single model has limited historical information,2MFIL algorithm proposes a multi-model knowledge distillation loss,which can constrain current model by using multiple historical models,and then alleviate catastrophic forgetting.4.Our approach models the few shot incremental learning as bilevel optimization problem.The upper optimization target is alleviating catastrophic forgetting,and the lower is learning a unified classfier.2MFIL algorithm uses alternate iterations to achieve better performance.A large number of experiments on public benchmarks have verified the superiority of our algorithm.