Research On Small Sample Named Entity Recognition Based On Transfer Learning

In recent years,deep learning has been highly regarded in the field of machine learning due to its excellent feature extraction capabilities and high accuracy performance.However,it still requires a large amount of labeled data for effective training.In many practical application scenarios,the lack of labeled data and the high cost of manual labeling make it difficult to solve the named entity recognition task,especially in specific fields,small languages,and other situations with a lack of labeling resources.Domain adaptation in transfer learning is an effective method that can adjust the source domain model parameters to adapt to the target domain,thereby solving the problem of a shortage of labeled data and difficult entity labeling in named entity recognition tasks.However,existing domain adaptation methods and smallsample named entity recognition still have the following problems: First,although the Maximum Mean Discrepancy method can reduce the difference in feature distribution between the source and target domains,it ignores the category information,which may limit the ability to recognize different entity types in the target domain.Second,differences in text data features and probability distributions between domains can cause the model to perform poorly in the new domain,potentially leading to domain shift problems.Third,in the past,methods for addressing the problem of small-sample classification often relied on token-based classification,which overlooked the importance of entity boundary information.This approach may be susceptible to interference from a large number of non-entity tokens.In order to address the problem of disregarding category information in previous crossdomain named entity recognition,this dissertation presents Class-Aware Cross-Domain Named Entity Recognition.By employing an improved maximum mean discrepancy method,the model minimizes the difference in joint probability distribution between the same category in different domains to achieve transferability,and maximizes the difference in joint probability between different categories in different domains to achieve discriminability.Moreover,the model introduces a contrastive learning module to learn domain-invariant features at the sentence level.The model learns not only domain-invariant features with category information,but also overall domain-invariant information.Contrastive learning has been proven to be an effective method for addressing domain shift problems.Therefore,this dissertation proposes Global Pointer based Domain Confusion Contrastive Learning for Named Entity Recognition.The model generates domain-challenging puzzles by searching for extreme directions that roughly point to the opposite domain.This strengthens the encoding capability of the model for both original samples and domainconfusion samples,enabling it to learn more robust feature representations through contrast with the original samples.Furthermore,the model introduces entity boundary information to mitigate the impact of non-entity tokens.By performing contrastive learning and computing KL divergence loss at the span level,the model captures critical information about entity boundaries,facilitating the learning of domain invariance and maintaining distribution consistency.In this dissertation,we conducted comparative experiments on multiple cross-domain named entity recognition models and ablation experiments on multiple modules using public datasets to verify the performance of the proposed models for the above issues.The experimental results show that the two proposed models can effectively address the above issues and improve the performance of cross-domain named entity recognition.