Research On Sentence Sentiment Classification Method Of Convolutional Neural Network Based On Forgetting Mechanism

Convolutional neural networks have been widely applied in natural language processing and other fields,and sentence sentiment classification tasks are one of the most common tasks in Natural Language Processing.Scholars have conducted a large number of experiments using deep learning neural networks in sentence sentiment classification tasks to prove that they can more effectively obtain context information in text data.At present,neural network models such as convolutional neural networks(CNN),recursive neural networks and recurrent neural networks(RNN)are usually used in sentence sentiment classification tasks.With the development of deep learning,the architecture combining neural network and attention has brought a major breakthrough to the development of sentence sentiment classification task,and the prediction accuracy has been greatly improved.These commonly used neural networks have their own advantages.For example,the convolutional neural network has excellent parallelism and can reach a high operation rate;whereas recursive neural networks,especially long-short-term memory network,can achieve good results when extracting long-distance dependencies between words.However,these neural networks still have some problems:(1)recursive neural networks are not very efficient at computational efficiency due to their recursive limitations;(2)Convolutional neural networks are limited by the size of their convolution kernels,which makes it difficult to capture the dependence between words in long-distance sentences.Based on this,design and implement a network structure that can effectively combine the advantages of the network model to achieve better performance is the mainly research motivation of this article.I have conducted the research to remedy convolutional neural networks deficiency in modeling dependency in long distance words and proposed an innovative method,a forgetting mechanism.We combine CNN and forgetting mechanism as a new architecture network,L-CNN.There are two type of context representation in L-CNN,local context representation and global context representation.For local context representation,we use common convolution operation to extract local information.And then the we would generate local context representation by the forgetting mechanism.This paper conducts experiments on four English text datasets,IMDB,SST-2,SUBJ and MR datasets.when measure the accuracy rate(ACC)and F1 with RNNs and CNNs.ACC increased by 1.78%,1.02%,2.14% and 2.5% respectively on the datasets,comparing to baselines.the experimental results show that L-CNN performs well in long sentences.We also compared L-CNN and CNN on both contextual word vectors and word vector.The results show that L-CNN have a better performance than CNN because of the gating mechanism,especially on long sentences.