Short Text Classification Algorithm Based On Temporal Convolution And Attention Mechanism

With the rapid development of Internet,the access to information is becoming more diverse and convenient and the total amount of network data is increasing explosively.How to extract valuable information from tons of data has become a popular research topic in artificial intelligence,and therefore the text classification technology is of great research value.Convolution neural network(CNN)performs well in many deep learning tasks,but the application of its deep architecture in text classification is controversial.In order to optimize classification accuracy,this paper improves deep CNN from four aspects and proposes new-type short text classification model.The main innovations are as follows:(1)Text information is discrete and sparse.In order to optimize the extraction of text global information and temporal information in CNN,and improve the high-dimensional feature representation in preprocess stage,we propose a novel model named DPTCN and a new-type mixed region embedding method for short text classification.In DPTCN,the pyramid pooling enhances the sampling for text global features and the temporal convolution reserves the sequential information between text.The experimental results prove that DPTCN is an innovative and effective proposal for deep convolution network in short text classification,and the mixed region embedding method improves the classification accuracy effectively without the need of extra corpus.(2)The key features are distributed unevenly among text information.In order to optimize the extraction of key information,we propose SE-DPTCN and ECA-DPTCN with the integration of channel attention mechanism into deep CNN,which improves the sensitivity of model to different channels,strengthens effective features,suppresses useless ones and eventually realizes the weighted calculation for convolution channels.SE module trains the weighted parameters through two fully connected layers in the excitation layer,which realizes the explicit modeling of interdependencies between channels.ECA module realizes cross-channel interaction by a fast 1-D convolution,which decreases the number of parameters in weight training.The experimental results prove that the integration of channel attention mechanism is an effective proposal to enhance the extracting ability of text key information and improve the classification accuracy of deep CNN.(3)The calculation of mean function in channel attention mechanism loses the variety of input features.In order to solve this problem,we propose FCA-DPTCN with the integration of multispectral attention module into deep CNN,which assigns different frequency components for each feature channel.FCA module remains the variety of features effectively and optimizes the attention weighting of input.The experimental results prove that multispectral attention module performs better on the preprocess of feature compression rather than average pooling and max pooling,and the integrated model has a better classification result.