Feature Matching And Diversity Transfer For Few-shot Image Recognition

Deep learning achieves great success in a variety of tasks with large amounts of labeled data for image recognition,machine translation,and speech synthesis.However,labeled data is not always massively available when annotation cost is too expensive or time is not allowed.By contrast,the human can learn novel concepts with only a few examples in a short time.Few-shot learning attempts to resolve this problem by training a model that classifies an unlabeled example based on a small labeled support set.In this thesis,two perspectives about few-shot learning are investigated,which are feature matching and diversity transfer.The main contributions of the thesis are summarized as follows:First,when only a few images with annotations are available for learning a recognition model for one category.The objects in testing/query and training/support images are likely to be different in the size,location,style and so on.Therefore,our method,called Cascaded Feature Matching Network(CFMN),is proposed to solve this problem.We train the meta-learner to learn a more fine-grained and adaptive deep distance metric by focusing more on the features that have high correlations between compared images by the feature matching block which can align associated features together and naturally ignore those non-discriminative features.By applying the proposed feature matching block in different layers of the few-shot recognition network,multi-scale information among the compared images can be incorporated into the final cascaded matching feature,which boosts the recognition performance further.The experiments for few-shot learning on two standard datasets,Mini Image Net and Omniglot,have confirmed the effectiveness of our method.Besides,the multi-label few-shot recognition task is first studied on a new data split of COCO.It further shows that the proposed feature matching network can focus on key features and improve the recognition precision when performing few-shot learning in complex images..Second,we find that the main difficulty of few-shot learning lies in the lack of intra-class diversity within insufficient training samples.To alleviate this problem,we propose a novel generative framework,Diversity Transfer Network(DTN),that learns to transfer latent diversities from known categories and composite them with support features to generate diverse samples for novel categories in feature space.The learning problem of the sample generation(i.e.,diversity transfer)is solved via minimizing an effective meta-classification loss in a single-stage network,instead of minimizing the generative loss in previous works.Besides,auxiliary task co-training over known categories is proposed to stabilize the meta-training process of DTN.We perform extensive experiments and ablation studies on three datasets,i.e.,Mini Image Net,CIFAR100,and CUB.The results show that DTN,with single-stage training,faster convergence speed and simple model structure,obtains the state-of-the-art accuracy.The proposed methods are general solutions for few-shot image recognition.The practical application scenes include: difficulty of collecting or annotating samples,insufficiency of computing resource,high demand for iteration speed,and so on.The researches in this thesis all try to solve the challenge in few-shot learning from different perspectives.The experiments on public benchmarks demonstrate that the few-shot learning task can benefit from the theory,models and algorithms proposed in this thesis.