Research On Methods Of Aspect Extraction And Aspect-level Sentiment Classification

With the development of Internet technology and the popularization of Internet applications,a large amount of text data have been generated on platforms such as social networks and e-commerce.These data often contain people’s opinions on related events,products and services,and the opinions have a certain sentiment tendency.How to make the machine understand the semantics of the text,identify the target object of the opinions,and detect the sentiment polarity towards the opinion target,has become a research hotspot in academia and industry.Among them,aspect extraction and aspect-level sentiment classification are one of key tasks.Although current deep learning-based methods have made significant progress in aspect extraction and aspect-level sentiment classification tasks,existing methods still suffer from many weaknesses due to the complexity of human language and sentiment expression.For example,the encoder cannot encode more useful information.The training process is highly dependent on the labeled data.The model is weak at modeling position invariance,local patterns,long-distance dependencies,and sequence information.Because of this,this thesis studies aspect extraction and aspect-level sentiment classification methods,it focuses on the following issues and proposes corresponding models.In aspect term extraction,how to encode the informative text representations,integrate surrounding word information,and model dynamic word meaning.In aspect category detection,how to implement unsupervised learning and fuse the context information and the noun information.In aspect-level sentiment classification,how to encode sequence information and semantic dependencies,as well as position invariance and local pattern sensitivity,and how to filter the noises generated by the attention mechanism.In deep learning models such as memory networks of aspect-level sentiment classification,how to achieve strong attention interaction,make word embeddings contain the context information,and utilize the information of each hop(layer).This research was funded by the National Key Research and Development Program of China(No.2018YFC0831306)and the Fundamental Research Funds for the Central Universities(No.2019YJS022).The main work and innovations of the work are as follows:1.An information-augmented neural network(IANN)for aspect term extraction is developed.To overcome the problems in the sequence-to-sequence methods that the sentence representation generated by the encoder lacks the informative information,the encoder is difficult to integrate surrounding word information,and the static word embeddings cannot model the dynamic meaning of words,an information-augmented neural network(IANN)is proposed.BERT is employed in the contextualized embedding layer of IANN to model dynamic word sense.In IANN,the encoder can encode the informative sentence representations and integrate the surrounding word information by the multiple convolution and recurrence operations.By extending the single-layer encoder to a multi-layer structure,the model can learn the higher-level semantic features that are more suitable for the specific task.In the decoding stage,an AO({Aspect,Outside})tag is proposed as the output label,it can improve the performance of the model.The experimental results on multiple datasets show that IANN outperforms the compared baselines.The experimental results verify the effectiveness of IANN in aspect term extraction.2.A multi-information fusion neural network(MIFNN)model for unsupervised aspect category detection is proposed.In aspect category detection,to overcome the problem that the existing models are highly dependent on the labeled data,and to solve the problems that the model did not exploit and merge the context and noun information,and not model sequence information and semantic dependencies in unsupervised learning,a multi-information fusion neural network(MIFNN)is proposed.MIFNN is a model with an autoencoder structure.An aspect auxiliary module is designed in the encoder of MIFNN to encode and fuse context information and noun information,it can provide general knowledge and aspect-related knowledge.To capture the semantic association between context words and aspect categories,a noun-aware attention mechanism is proposed,it can generate the contextual sentence representation related to aspect categories.To capture the semantic relationship between the information fusion representation and the words in a sentence,a fusion information attention mechanism is developed in the encoder of MIFNN.The decoder of MIFNN reconstructs the sentence representation generated by the encoder through learnable aspect embeddings.Experimental results on multiple datasets show that MIFNN outperforms the compared baselines on unsupervised aspect category detection.The experimental results verify the importance of MIFNN in unsupervised aspect category detection.3.A gated alternate neural network(GANN)model for aspect-level sentiment classification is developed.To overcome the problems that recurrent neural networks lack the position invariance and are insensitive to local patterns,convolutional neural networks are not good at modeling sequence information and capturing long-distance dependency information,and the attention mechanism may introduce some noises,a gated alternate neural network(GANN)is proposed.To better capture aspect-related information in a sentence,the sentiment clue is developed.A gated truncation RNN is designed to divide a sentence into multiple sentiment clues,it can encode the relative distance between each context word and the aspect term,sequence information,and semantic dependencies within a sentiment clue.To obtain a more accurate sentiment clue representation,a gated filtering mechanism is designed to control the information flow during the encoding process.GANN employs convolution and pooling mechanisms to capture local key sentiment features and obtain position invariance.Experimental results on multiple datasets show that GANN outperforms the compared baselines.The experimental results verify the effectiveness of GANN.4.A recurrent memory neural network(ReMemNN)model for aspect-level sentiment classification is proposed.To overcome the problems that the binary attention mechanism cannot model the strong interaction relationship between aspect terms and the context,the information of each hop in the multi-hop model is not fully exploited,and the word embeddings lack the specific semantic information from the context in deep learning models such as memory networks,the recurrent memory neural network(ReMemNN)is proposed.In ReMemNN,an embedding adjustment learning module is designed,it can make the word embeddings learn semantic information from the context.To model the strong interaction relationship between aspect terms and the context,a multi-element attention mechanism is proposed,it can generate more accurate attention weights and informative aspect-dependent sentence representation.To make full use of the aspect-dependent sentence representation of each hop in the model,an explicit memory module is designed and separately generates the sentence representation for a multi-element attention mechanism and prediction of sentiment polarity.Experimental results on multiple datasets show that ReMemNN achieves better performance than the compared baselines.The experimental results verify the effectiveness of ReMemNN.