Research And Application Of Knowledge Extraction Based On Few-shot Learning

With the maturity of internet technology,a huge amount of textual data has been generated in the information age.How to effectively obtain structured knowledge from unstructured text data is particularly important.Knowledge extraction technology is an important basic task in the field of natural language processing.It is an important method for the structured processing of mass data.Currently,knowledge extraction is mainly divided into named entity recognition,relation extraction and event extraction.Traditional supervised knowledge extraction methods are driven by large-scale and high quality datasets,which can often get objective results.However,high quality and large-scale labeling datasets require expensive costs.In many areas where data resources are insufficient,the model effect of traditional supervised methods is not ideal.Therefore,many researchers have begun to explore the application of few-shot learning to knowledge extraction tasks,and create a lot of excellent work on the few-shot knowledge extraction tasks.But there are still some problems.Firstly,in the few-shot named entity recognition task,the model has insufficient classification ability in the face of confusing tokens.The second aspect is the few-shot relation extraction task.On the one hand,in the few-shot relation extraction method based on the prototypical network,there is a lack of use of relation representation in the construction of prototype points.On the other hand,the feature representation of the entity pair by the pre-trained language model is more biased towards the rationality of the sentence than the internal connection between the entity pair.This thesis starts with the fewshot named entity recognition task,the impact of confusing token representation on the model is reduced by adding token contrastive learning tasks,and the effect of few-shot named entity recognition is improved.Then,aiming at the problems in the few-shot relation extraction task,the dynamic prototypical network construction method based on hybrid representation and the pre-trained language model training method based on contrastive learning are adopted to improve the effect of few-shot relation extraction.The specific research contents are as follows:(1)In the token-level few-shot sequence tagging task,existing methods did not perform well in the two complex situations of unknown domains and noisy token representation.To solve this problem,a few-shot named entity recognition method based on joint multiobjective optimization(MONER)is proposed.Based on the nearest neighbor classification algorithm task,MONER joints the loss of metric learning and the contrastive learning task.It optimizes the network parameters to improve the performance of the model.The results on two sub-datasets INTRA and INTER based on FEW-NERD show that MONER achieves a better average F1 score than the baseline model,which verifies the effectiveness of MONER in few-shot named entity recognition task.(2)Aiming at the problem that the traditional prototypical network cannot effectively solve the prototype deviation caused by noise data,a few-shot relation extraction method based on fusion hybrid representation is proposed.In this method,the relation representation and entity mention are introduced as auxiliary information,and the prototype points of relation categories are dynamically constructed to improve the processing ability of the model to noise data.Therefore,a more accurate prototype discriminant representation is obtained.The experimental results show that on the international public datasets Few Rel1.0,the few-shot relation extraction method with fusion representation achieves higher accuracy than the baseline model under different subtask settings.It is verified that the proposed method can effectively use auxiliary information to alleviate the problem of prototype deviation to obtain a better effect of relation extraction.(3)Aiming at the limitation of relation feature representation existing in the current pre-trained language model in the task of few-shot relation extraction,a method of pretrained model for few-shot relation extraction based on contrastive learning is proposed.The BERT model is optimized and trained through distance supervised labeled data,and fine-tuning in downstream tasks to achieve better relation extraction results.Adding the auxiliary task based on contrastive learning,and optimizing the bias of the pre-trained model’s relationship representation according to the difference between the positive and negative samples of the instance and prototype.The experimental results show that on the Few Rel1.0 datasets,the method of extracting pre-trained models based on contrastive learning with few-shot relation achieves higher accuracy than the baseline model,which verifies that this method is effective in few-shot relation extraction task settings.