Multitask Learning For Sentiment Analysis

Sentiment analysis has been a well-studied domain,there are currently two methods frequently used,the lexicon-based method and the machine learning based method,and the deep learning based method is the most popular method in the machine learning based method.The lexicon-based method relies heavily on the quality and coverage of sentiment lexicons,while the deep learning based approach requires a large amount of labeled data.The model is prone to overfitting if the dataset is too small,multi-task learning improves generalization by leveraging the domain-specific information contained in the training signals of related tasks,to some extent,it alleviates the problem of overfitting.After our investigation,we found that existing models have a few shortcomings:(1)Most multi-task learning models use LSTM for sentiment analysis,the last hidden state of LSTM is often used as the representation of the input,while this representation tends to ignore the features at the front,thus is biased.(2)In the case that there are multiple emotional words in the text and the grammatical structure is complex,the prediction of existing methods can be random.(3)The calculation of the current state in LSTM depends on the last state,so it's very inefficiency to be parallelized.(4)The existing methods often divide the feature space into private space and shared space,and each task has an independent private feature space which consumes a lot of memory when the number of tasks is large.To solve those problems,we have done the following work:(1)We proposed the AASPMTL model that uses the attention mechanism to solve biased representation,and the attention mechanism can be used to visually analyze the model.(2)The lexicon-based model is good at solving the complex grammatical structure,so we use the VADER model to help improve multi-task learning.(3)For the parallel inefficiency and memory consuming problem,we proposed the DT-MTL model that uses self-attention only which is parallel efficiency and does not use extra memory.We tested our two models with 16 sentiment analysis datasets,and the results show that the AASP-MTL has lower average error rates than the ASP-MTL model,the average error rates of the DT-MTL model is lower than the ASP-MTL model but slightly higher than the AASPMTL model.The parallel acceleration ratio of the DT-MTL model is far superior to the AASPMTL model.So when the dataset is small,we should use the ASP-MTL model because it has a lower error rate;when the dataset is big,we should use the DT-MTL model because it trains faster.