Research On Bert-based Named Entity Recognition

With the rapid development of information technology and the widespread application of information equipment such as computers,human society has entered the information age.At the same time,the accompanying network security issues have also brought potential threats and losses to national security and social development.How to ensure cyberspace security in the information age has become an important issue that academia and industry are concerned about.In recent years,the use of knowledge representation methods to model key elements such as vulnerabilities,assets,and attack activities in the field of network security,and to provide knowledge support for network security situation analysis,emergency response and other links,has gradually become an effective research method.Especially with the rapid development of related technologies represented by Google Knowledge Graph,the network security field has begun to introduce knowledge graphs to express network security knowledge,and the realization of accurate named entity recognition is an important prerequisite for the construction of knowledge graphs.Related research has found that traditional named entity recognition methods have some shortcomings in dealing with the specialized field of network security.For example,named entity recognition algorithms based on Hidden Markov Models are strictly used because most of the network security statements do not conform to Markov.The result is poor recognition of named entities.In response to the above problems,this paper focuses on solving the problem of named entity recognition accuracy in the field of network security.The main tasks include:Firstly,this paper analyzes the shortcomings of adopting the Bert model directly in the domain of network security named entity recognition.In view of the problem of large parameters of Bert model,an improved Bert algorithm is designed.Bert algorithm is suitable for medium length text.In the field of network security,named entity recognition is mainly input by sentence.When using the Bert model directly,it has a large number of parameters,leading to long training time and low efficiency.In this paper,by sharing the self-attention layer parameters in encoder,the parameters of the model are reduced and the lightweight improvement of the Bert model is realized.The experimental results show that the parameters are reduced by 19.4%,while the performance is basically unchanged.Secondly,this paper uses the BIOES model to mark the network security data,and divides the named entities into three categories: assets,vulnerabilities and attacks.In view of the characteristics of large scope and noise in the field of network security,this paper designs BertBi LSTM-CRF model based on improved Bert algorithm.The named entity recognition in network security field is realized by the improved Bert algorithm preprocessing,encoder,conditional random field and so on.Finally,combined with the above algorithm,this paper designs a named entity recognition prototype system in the field of network security.Specifically,the output of Bert module is used as input of the Bi LSTM module;the output of the previous module is decoded by the Bi LSTM module;the output of the Bi LSTM module is used as the input of the conditional random field module,and the output of the module is decoded by the conditional random field module;finally,the named entity recognition in the field of network security is completed.The model has strong feature extraction ability,and could achieve good results in the field of network security named entity recognition.In conclusion,this paper improves the Bert model by parameter sharing,which makes it short training time and high efficiency when it is adopted for named entity recognition in network security field.Secondly,the Bert-Bi LSTM-CRF model is designed,and the named entity recognition in the field of network security is designed by the improved Bert model,encoder and conditional random field.Finally,this paper designs a named entity recognition prototype system for network security.