Research On Text Classification Based On Word2vec And Convolutional Neural Network

Based on the development of Internet technology,all types of information resources grow on a large scale,among which textual data always take an important position.Relying on manual methods to manage and classify these huge amounts of textual data consumes a lot of time and manpower,and it is also difficult to achieve.Thus,how to effectively manage text resources becomes an urgent problem to be solved,which promotes the rapid development of automatic text classification technology.At present,automatic text classification has been widely used in fields such as knowledge discovery and information retrieval.As Internet text data will continue to increase,automatic text classification technology will play an increasingly important role in the future.Text classification involves many techniques in the fields of machine learning,natural language processing and data mining.Many factors affect the performance of text classification.The main factors include text preprocessing,text representation,as well as classifier selection and optimization.Among them,the primary problem in text categorization is how to extract features and how to reasonably represent texts by computers.Therefore,exploring efficient text representation methods is of great significance for the study of automatic text classification technology.Traditional text representation methods,such as boolean model and vector space model,have problems of data sparseness and curse of dimensionality,which leads to poor generalization of the model.With the rapid development of machine learning and deep learning technologies,researchers have begun to construct text representation models using various neural networks,and map texts to low-dimensional continuous vectors through neural networks improving the representation performance of the model.However,the current neural network text representation model also also has some problems.First of all,the classification ability of the neural network is lack although it grasps semantic information better in text representation.Secondly,the existing method for extracting text features using a convolutional neural network mainly uses the length of the longest text in all the texts of the data set as a reference,and the rest of the texts smaller than this length are filled with special characters.This leads to the introduction of too many invalid characters in the shorter text,affect the original text information and reduce the performance of the model when the text is categorized.This paper systematically elaborates text representation and feature extraction in text classification technology,analyzes the existing text representation model,improves the shortcomings in the current text representation model.The main ideas of this paper are as follows.（1）A single text representation model（CD_STR）and a fusion model（CD_MTR）with classification capabilities are proposed,among which the CD_STR model incorporates the improved models of TF-IDF and word2vec.First,improve the TF-IDF weight calculation method.Then,construct a single text representation model to make the text representation more efficient as word2vec can better represent the semantic information of the feature words.On the basis of the improved CD_STR,a CD_MTR model with better classification capabilities was proposed.And on the analysis of the existing text representation model,the latent semantic index（LSI）,the TF-IDF weighted vector space model（T_VSM）and the single-text representation model CD_STR are merged to make a fusion model with better classification capability.By testing on two data sets,it is verified that the performance of the two models,CD_STR and CD_MTR,is greatly improved compared with other text representation models.The micro-average F₁ value and the macro-average F₁ value obtained by the CD_MTR model are increased by 4%to 5%compared with the commonly used word2vec represented text vector accumulation and then re-averaging method.（2）A text classification method（MTR_MCNN）based on fusion model CD_MTR combined with convolutional neural network is proposed in this paper.First,improve the network structure of the convolutional neural network model,and design different sizes and numbers of convolution kernels,so that the text features can be extracted from different perspectives.Then,train the text represented by the CD_MTR model and input it in the improved convolutional neural network,then the automatic abstraction of multiple local abstract features is further completed.After the pooling operation,the high-level text features are obtained.Finally,it reaches the full connection layer and softmax regression layer for text classification.The MTR_MCNN method can solve the problem that the length of a longest text in a dataset is used as a benchmark in commonly used methods,and texts smaller than this length are filled with special characters to affect the original text information expression,and the dimensions of the input matrix can be reduced.Further improve the training speed of convolutional neural networks.By testing on two data sets,the MTR_MCNN method has greatly improved the classification accuracy compared with other classification methods,and further improves the accuracy of text classification compared to the CD_MTR model proposed in this paper.