| With the advent of the era of big data,massive information is flooding all aspects of social life,and how to extract important data from it intelligently using natural language processing has become a problem worth studying.As one of the key tasks of information extraction,named entity recognition can identify and classify entities from unstructured text,thus helping researchers to complete data extraction accurately and efficiently.In this paper,we conduct a study on Chinese named entity recognition methods and applications based on pre-trained models.In this paper,we first construct the ELECTRA-FLAT-CRF model based on ELECTRA(Efficiently Learning an Encoder that Classifies Token Replacements Accurately)model and FLAT(Flat Lattice Transformer)model,combining the word fusion mechanism with the pre-trained model so that the model can fully utilize the unique semantic information in Chinese text;based on it,we propose the MEFC(Multi-Task-Learning ELECTRA-FLAT-CRF)model based on multi-task learning,using the namely sentence level named type prediction task and the namely entity boundary prediction task respectively to help The NER task learns the rich semantic information contained in the text from multiple perspectives.In order to implement MEFC model functions with less hardware resources,this paper proposes a compression method of MEFC model based on knowledge distillation-"Distil-MEFC" model,and further explores the performance enhancement method of distillation model by combining with adversarial training algorithm.In this paper,ablation experiments and comparison experiments are conducted on the proposed model in four publicly available datasets.The experimental results show that the F1 value of ELECTRA-FLAT-CRF model compared to both ELECTRA-CRF model and FLAT-CRF model with different degrees of improvement.The MEFC model has the best performance.The Distil-MEFC model is able to achieve a higher compression ratio with a lower performance loss rate,and it improves the inference speed to a certain extent,and obtains a better model lightweighting effect.The adversarial training can further improve the model distillation on larger data sets.Named entity recognition in the field of administrative work reports can help governments,enterprises and individuals to quickly understand social and economic information,and thus provide a strong basis for relevant decisions.However,since there are few studies on administrative work reports and there is no publicly available dataset in this field,this paper constructs a named entity recognition dataset for this field based on100 administrative work report documents collected in this project in order to conduct relevant research.The model parameters for this dataset were then obtained using the model proposed in this paper trained on this dataset.Finally,the administrative work report entity recognition tool software was designed and implemented based on the distillation model,which has five core functions: entity recognition,model training,data processing,history management and user management,and can help researchers extract information from administrative work report texts accurately and efficiently through a visual interface. |
