Few-shot Learning: Theories And Methods

The emergence of deep learning promotes the development of Artificial Intelligence.However,different from human intelligence,although the fitting ability of deep learning is strong enough,the reasoning ability is rather weak.Hence,in the case of insufficient training samples,deep learning lacks the ability of fast learning like human beings.In certain practical applications,it is costly to get training samples and it is common that the training samples are insufficient.To this background,the concept of few-shot learning appears.The target for few-shot learning is to improve the learning ability of machine learning models under insufficient training samples.Enhancing the generalization ability and the reasoning ability as well as using training data more efficiently are key factors to real AI,which need to be tackled in few-shot learning.In this paper,we mainly focus on few-shot learning for deep learning models.We theoretically promote few-shot learning from three aspects: the classification layer,the representation layer and the input layer.Finally for a practical application scenario of black-box adversarial attack,we improve the attack efficiency from a few-shot learning perspective.We conclude our contributions as follows:· From the perspective of the classification layer,we propose few-shot learning algorithms based on visual analogy.The ability of fast learning of human beings is attributed to the visual analogy mechanism.Humans are able to learn a new concept from previously learned similar concepts through generalization.Hence,we propose a Visual Analogy Graph Embedded Regression(VAGER)algorithm on the classification layer of deep learning models,to simulate the mechanism of the visual analogy of human beings and further enhance the generalization ability of deep learning.· From the perspective of the representation layer,we propose the optimal representation for few-shot learning.The traditional representation layer of deep neural networks is fit for learning through large-scaled training samples,while it may perform worse on few-shot learning.Inspired by the Minimum Variance Unbiased Estimation(MVUE)theory,we propose the Discriminative Variational Embedding(DVE),a new representation for few-shot learning which substantially enhances the efficiency of the few-shot learning for deep models.· From the perspective of the input layer,we propose the base class selection problem in few-shot learning for the first time in the world.The selection of base classes is very important to the generalization ability of the model in the few-shot learning.Reasonable base classes can greatly improve the learning efficiency of novel classes.Hence we propose a base classes selection algorithm based on Similarity Ratio,which constructs the base dataset with strong generalization ability from a broad of candidate classes.· From the perspective of practical application,we apply few-shot learning in a practical black-box adversarial attack problem.We propose the Eigen Black-box Attack(Eigen BA),utilizing the gradient information of a pre-trained model to infer the optimal perturbation direction in black-box adversarial attack.Experiments show that the proposed algorithm achieves more effective attack by reducing the query number of the black-box model.