Text Classification Based On Attention-Based C-GRU Model

With the rapid development of Internet and the wide application of information technology,massive text data has emerged in the current era.How to manage these text data effectively and discover the valuable information has become a huge challenge in the era of big data.As the key technology of text information processing and the classic NLP task,text classification has always been the focus in academic field.With the remarkable achievements of deep learning in image processing,speech recognition and other fields,it has been proved that deep learning belongs to an effective tool to extract high-level representation of sentences or text in NLP task.However,the single deep learning model has the problems of ignoring the content order and the effect of keyword differentiation.Therefore,it is a hot research topic to design the text classification algorithm based on the deep learning hybrid model,which combines the advantages of each model and extracts the high-level features.Based on the analysis of text classification technology and deep learning model,this paper focuses on how to design deep learning hybrid model reasonably and solve text classification problems by extracting high-level features using the hybrid model.The main research work of this paper can be summarized as:(1)A deep learning hybrid model named Attention-Based C-GRU is proposed to solve the semantic missing problem by exploring the possible combination and improving performance.The model is composed of input layer,C-GRU feature extraction module,Attention mechanism weighting module and Softmax classifier module.The proposed model combines the improved convolution layer with GRU model in a unified architecture to form a two-level feature extraction module,which takes advantage of the CNN and GRU to improve the efficiency of text classification.(2)The traditional CNN model ignores the context order,leading to the lack of textual semantics.For this problem,we propose an improved convolution layer algorithm.We extract the convolution layer from the CNN model and improve the structure of the convolution layer by defining region sequence vector RSV.The abstract feature representations have been produced for different position word vectors,which form more accurate feature representations by combining multiple local features.The feature sequence improves the efficiency of feature extraction and enriches the semantic by maintaining the word order relationship of the texts.(3)The selection randomness of the Local-Attention window dimension results in the problem of keyword loss or semantic redundancy.For the problem,the selection strategy of window optimal dimension is proposed.This strategy is used to select the range of Local-Attention reasonably,which improves the optimization efficiency of Local-Attention and enhances the effect of keyword differentiation on text categorization.In order to prove the validity of Attention-Based C-GRU model,and verify the optimization of improved algorithms in convolution layer and the Local-Attention mechanism selection strategy.In this paper,a set of comparative experiments are set up on the Chinese and English text categorization corpus.By comparing with the baseline models and the state-of-the-art methods on several corpus,the result shows that the proposed model has a better effect on text classification and the improved methods can improve the efficiency of feature extraction.