| Few-shot learning aims to learn a model with good generalization capability such that it can be readily adapted to new unseen classes by accessing only one or few examples.This has important practical significance in real applications,because new samples from new classes are usually difficult to obtain,and on the other hand,sample annotations for some applications are also very difficult.In addition,the extremely limited samples per class can hardly represent the class distribution effectively,making this task truly challenging.There are mainly three types of methods in few-shot learning,namely,data aug-mentation based,meta-learning based,and metric based.This paper mainly focuses on the metric based methods,analyzes the challenges in this direction,i.e.,transferable knowledge,image representation,concept representation,relation measure and model robustness,and proposes the corresponding solutions.Specifically,four novel methods has been proposed in this paper as follows:(1)A covariance metric network(CovaMNet)is proposed.The existing few-shot learning methods mainly focus on using first-order statistic to represent concepts,which is not very effective in few-shot setting.Instead,the second-order statistic is employed to represent concepts,because the second-order statistic contain more information than the first-order statistic,and will benefit more for few-shot learning.Specifically,we adopt a covariance matrix which contains the second-order statistic information to represent a concept.Furthermore,rich deep local descriptors are introduced into few-shot learning at the first time to calculate covariance matrix.Upon the covariance representation,we further define a new deep covariance metric to measure the consistency of distributions between query samples and concepts.Meanwhile,a simple theoretical analysis is also provided to explain the effectiveness of this metric function.(2)A contrastive distribution metric network(CDM)is proposed.CovaMNet has firstly explored distribution-based representation(i.e.,the second-order covariance representation)for concepts,but it ignores the underlying distribution of query sam-ples.In addition,although CovaMNet has taken the second-order information into consideration,it neglects the equally important first-order information.Different from CovaMNet,both first-order and second-order information are considered in CDM.Also,we adopts distribution based representations for both query samples and concepts from the perspective of distribution.In the relation measure,we employs a distribution based metric function,i.e.,2-Wasserstein distance,to directly measure the distribution consistency between query distributions and concept distributions.Furthermore,a dis-tribution based triplet is proposed to make the distribution distance more discriminative(3)An image-to-class measure based deep nearest neighbor neural network(DN4)is proposed.Although CovaMNet and CDM have tried to use deep local descriptors instead of conventional image-level features as the image representations,they still sum-marized these local descriptors,i.e.,calculating covariance representations.Inspired by NBNN[1],we know that the quantizing operation(e.g.,global feature or covariance representation)will lose considerable discriminative information,which is not recov-erable in the few-shot setting.Therefore,DN4 employs the raw non-summarized deep local descriptor pool to represent query samples and concepts,respectively.More-over,instead of the conventional image-to-image measure,we adopts an image-to-class measure,which can enjoy the exchangeability of visual patterns(4)A multi-level distribution based adversarial training(MDAT)is proposed The existing few-shot learning methods only focus on few-shot classification on clean samples,while ignoring the robustness and security of the models.However,conven-tional machine learning models and deep models are very vulnerable to adversarial attacks,which are perceptually indistinguishable.Therefore,in this paper,a new topic-defensive few-shot learning is introduced at the first time.In this new topic,a episode-based adversarial training mechanism is adopted to help the transfer of adversarial defense knowledge.Moreover,on the adaptivity of task defensive,a task-conditioned distribution constraint is also proposed to improve the robustness of models. |
