Augmented Prototypical Networks For Few-shot Learning

Few-shot learning(FSL)aims to solve a variety of tasks including relation extraction based on only a small amount of sample data in data-scarce domains.FSL is characterized by cross-task,small amount of data,and high learning difficulty.Meta-learning is the most popular and significant approach in FSL,because it is able to learning rapidly and has strong ability of generalization.Meta-learning is a method which learn the association and moomon features between different tasks on the task space,and then use the learned knowledge as the prior to model,guide and optimize the learning process of new tasks.It can accelerate the training speed and is easy to generalize.Prototypical network which is a metric-based meta-learning method is the best among exiting methods for its simplicity and efficiency.However,current researches on FSL are mainly focused on the image domain,and relatively little research has been conducted on NLP domain.In this paper,we investigate the task of few-shot relation extraction,where relations between mentioned entities in unstrctured text are extracted to compose triple information.To enhance the performance of existing models,we propose a novel meta-learning method which is based on prototypical network.The specific researches in this paper are as follows:1.We investigate the main problems and the current research status of the few-shot relation extraction tasks,and study three sub-tasks:in-domain task,cross-domain task,and cross-sentence task,respectively.Then,an improved prototypical network is proposed to enhance feature representation and improve the ability of generalization rapidly.For the in-domain task,we propose to incorporate the Transformer module to improve the ability of feature representation and capture long-range dependencies.For cross-domain task,we introduce pre-trained BERT to reduce the dependence of downstream tasks on domain data while improving the word representation.For cross-sentence task,since a pair of entities to be queried are located in different sentences,we perform dependency parsing for co-reference resolution.Above all,we propose the improved model TPN\TPN_BERT,and describe them in formalization.2.Since the current cross-sentence relation extraction dataset is rare and hard to satisfy our experiments,we clean the data which come from several open source and then construct the cross-sentence dataset called Few SP for FSL,and the single-domain dataset Few Rel and the cross-domain dataset Few Rel2.0 are repartitioned.Then,we compare our proposed model with existing few-shot models on the above three datasets which is correspond to subtasks,and analyze the advantages and disadvantages of each model to illustrated the superiority of our proposed model.Lastly,we show the visualization of relative distance to illustrate the effectiveness of our feature extractor Transformer,and the number of Transformer blocks and attention heads are varied respectively to observe their effects on the performance of our model.Compared with existing methods,our proposed model TPN\TPN_BERTachieves a balance between the computation and the performance on all of the three domain tasks.When compared with the methods without pre-training,our method has state-of-the-art performance with a large accuracy improvement.As above,our experiments fully demonstrates the superiority and robustness of our proposed model TPN\TPN_BERT.