The Research And Application Of Neural Network In Short Text Classification

With the rapid development of computer technology and the Internet industry,social platforms such as WeChat,Weibo,Twitter and Facebook,etc.have become more mature as the representatives of Internet products.People can share their views and opinions on social platforms anytime and anywhere.In addition to general users,more and more self-media and official media have also settled on these platforms to publish news.These social platforms are playing an increasingly important role in people's daily lives.Not only they can promote mutual communication between individuals,but they can also help people to spread and obtain news information more quickly.This has been particularly obvious in recent years.A large number of active users are constantly joining in,leading to an explosive growth of text data in social platforms.Hundreds of millions short text data are generated on these platforms every day,such as news feeds,user comments and chat records,etc.,which contain a lot of valuable information.How to use the short text classification technology to mine valuable information behind the data,has great research significance and application value.The most important part of short text classification is how to extract and represent short text features.Traditional text representation methods usually use static language models to train word vectors,and cannot adjust the input word vector as needed.At the same time,the relationship between short text context,hierarchical structure and part-of-speech are ignored,and problems such as sparse features,insufficient semantics,and too high dimensions lead to unsatisfactory classification results.Based on this,the thesis studies how to represent short text features more reasonably and effectively.Firstly,proposing a short text classification model based on ON-LSTM and Hierarchical Attention Networks mechanism(ON-LSTM-HAN).The BERT model based on the transformer architecture is used to pre-train the corpus dynamically to obtain semantic information,and the word vector is used as the input layer's data.Then the word vector though the ON-LSTM model to judge it's hierarchical structure to obtain the ordered information of the neurons and the hierarchical word vector representation.The analysis of hierarchical structure based on the unidirectional ON-LSTM,it shows that the current hierarchical analysis mainly relies on historical information.Therefore,the historical information contribution rate is introduced to adjust the influence of historical information.By adjusting the contribution rate to optimize the word vector,using the Hierarchical Attention Networks mechanism to assign the weight for the word vector to highlight the main information,then get the final sentence vector representation.Secondly,proposing a short text classification model based on improved part-of-speech information and ACBiLSTM.The BERT model based on the transformer architecture is used to pre-train the corpus dynamically to obtain semantic information,while introducing optimized parts of speech to improve it,the obtained word vector based on part-of-speech information is used as the information of the input layer.The features of the word vector are preliminarily extracted through the convolutional layer of the convolutional neural network model(CNN model),Then,the intermediate vector with context time series information is generated by the bidirectional long-short-term memory network model(BiLSTM model).Finally,it combines the attention mechanism to highlights the key information,and obtains the final text feature representation.In order to verify the effectiveness of the proposed model,for short text classification models based on ON-LSTM and Hierarchical Attention Networks mechanism,the thesis uses the SemEval2014 Task 4,SemEval2017 Task4 datasets as the data set;For short text classification models based on improved part-of-speech information and ACBi LSTM,the thesis uses the public text classification data of Fudan University,Sougou news corpus provided by Sogou Lab,and a subset of the Sina News text classification data set provided by the Tsinghua NLP team as the data sets.The experimental results show that,the models proposed in the thesis have theoretical significance and applicational value,they can optimize the short text feature expression,improve the accuracy and F1 value effectively.