Research On Deep Learning Text Classification Method Based On BERT Model

Text classification,as an important research direction in natural language processing,aims to classify text content into several categories according to different features and certain criteria.With the increasing demand for massive text data processing in information-based society and the wide application of text classification in real life,it is significant to conduct research on it.The existing research on text classification based on deep learning has flourished and achieved many results,but there are still several problems that affect the accuracy of text classification.First,the text content is sparse and the text features are few,so the model cannot identify the key semantic information.Second,the models focus too much on local information and ignore the global information.Third,the long text semantic logic is complex,and inaccurate semantic understanding also leads to inaccurate text classification.To address the above problems,the research content and main contributions of this paper are mainly three points as follows:1.To address the problems that existing models have poor ability to recognize key semantic information and may ignore global information by focusing too much on local information,this paper proposes a text classification method based on the BERT-AttTextCNN model.First,this paper uses the BERT model to obtain high-quality sentence vectors.Meanwhile,the design of this paper combines attention mechanism and TextCNN network to extract and enhance local key features.Then,the model in this paper splices the hidden output of BERT with the pooled output of TextCNN to strengthen the judgment ability and robustness of the model.Experiments show that this paper’s method is tested on THUCNews,SST-2,and content security datasets,and the accuracy reaches 94.8%,94.6%,and 99.4%,respectively,which is better than the classical model.The model in this paper has good generalization ability while improving the accuracy.2.In view of the complex semantic information in long texts and the problem that semantic understanding is prone to deviation,this paper proposes a semantic matching model based on BERT-Bi GRU.First,input the sentence pair into the BERT pre-trained language model to obtain accurate semantic features of the sentence pair.Secondly,a gating selection mechanism is designed to dynamically fuse the features containing contextual semantic information obtained through Bi GRU and the key semantic features extracted through the TextCNN network,so that the model can understand long text semantic information more accurately.Experiments have proved that the performance of the model in this paper is better than that of the classic model on the community question answering dataset,with an average improvement effect of 41%,which is remarkable.3.In this paper,the deep learning text classification method based on BERT model proposed in Chapter 2 is applied to the community intelligence security degree assessment system.By analyzing the textual characteristics of community intelligence information and the needs of the system for intelligence information assessment,this paper constructs a content security degree dataset for training the text classification model.Experiments prove that the text classification model proposed in this paper performs well on this system.At the same time,the dataset constructed in this paper makes up for the lack of content security dataset in the Chinese domain.In summary,this paper improves the accuracy of text classification by optimizing the traditional deep learning-based text classification model,and the model has good generalization ability.Secondly,this paper solves the problem of semantic interaction between sentences by designing a semantic matching model to make long text semantic information understanding more accurate.Finally,the method in this paper is applied for the first time to a practical scenario of community intelligence security assessment,with good results.