Text Sentiment Analysis Based On Multi-head Attention Mechanism

Due to cognitive growth and rapid development in social media platforms such as Facebook,Twitter,etc.They have become a powerful source of communication between users to share and express their opinions and feelings as views and opinions towards a particular article or topic,resulting in a massive amount of unstructured data.Thus,business organizations and companies need to study and process these feelings and viewpoints to inspect and investigate data to construct business insights.Therefore,analyzing textual sentiment has become a challenging task for researchers;this prompted many studies to focus on the domain of textual sentiment analysis.More recently,studies have shown that deep learning approaches such as Recurrent Neural networks(RNN),Convolutional Neural networks(CNN),and Transformer-based methods offer promising performance in the field of sentiment analysis.Although CNN has the ability to extract high-level features via convolutional and max-pooling layers,it cannot learn a sequence of correlations efficiently.On the other hand,Bidirectional RNN employs two RNN directions to improve the extraction of long-term dependencies.Thus,it has no ability to extract local features in parallel.Besides,Transformer-based such as Bidirectional Encoder Representations from Transformers(BERT)have achieved promising performance in sentiment analysis tasks due to their ability to control in computational resources necessary to fine-tune.However,these approaches still face an overfitting issue on small datasets.In our study,we leverage the ability of deep learning approaches to address some issues faced by RNN and CNN models,including the high dimensional feature distance,overfitting,and the sparse text data.We propose a hybrid model for sentiment analysis tasks that combines the deep neural networks(DNN)and multi-head attention mechanism(MHAT),called(DNN-MHAT)for Sentiment polarity detection.In DNN–MHAT,we first applied a global vector for word representation(Glo Ve)to create word vectors automatically as the weights in the embedding layer for extracting meaningful information from text data.We also design an improved deep neural network to capture the text’s actual context and extract the local features of position invariants by combining recurrent bidirectional long short-term memory(Bi-LSTM)units with a convolutional neural network(CNN),which enhances the learning ability representation of the text.Then,we apply a multi-head attention mechanism on the output of the Bi-LSTM layer to capture the words of sentiment in the text to long space and encode dependencies among words,which adds a different focus to the information produced from the hidden layers of Bi-LSTM.Finally,a global average pooling is applied for transforming the vector into a high-level sentiment representation to avoid model overfitting,and a sigmoid classifier is used to carry out the sentiment polarity classification of texts.The DNN–MHAT model has been conducted on two types of datasets: four long review datasets,including Kindle,APP,CD,and,Electronics,and two short tweet datasets,including Airline tweets and sentimet140.The results of the experiments illustrate the effectiveness of the DNN–MHAT model,which achieved superior performance compared to the state-of-the-art baseline methods based on short tweets and long reviews.The experiments compared the DNN–MHAT model with the existing five state-of-the-art DNN baseline deep learning techniques based on sentiment analysis and text classification datasets.The DNN–MHAT proposed model outperformed the existing methods in terms of popular performance standards in NLP and the domain of sentiment analysis.