Research On Deep Learning Algorithm And Data Augmentation In Text Classification

In recent years,artificial intelligence has witnessed the rise of deep learning.Utilizing the advantages of network depth,deep learning has shown excellent performance in many areas,and in some areas it has surpassed the level of humans.However,with the gradual progress of research,deep learning has also begun to face many challenges,mainly including 1): As the depth of the network continues to increase,the problem of gradient vanishing gradually becomes more prominent in deep learning,which not only makes the training of deep learning more difficult,Limitations and even worsened its performance;2)Deep learning is particularly dependent on the support of massive data,however,the process of preparing massive data for it is extremely time-consuming and labor-intensive,making it generally face the problem of insufficient data.Aiming at the above two problems,this paper aims to improve the performance of deep learning in text classification as the overall goal,and proposes the following three major innovations to respond:1): To address the problem of gradient vanishing in deep learning,based on two new types of deep neural network architectures,Highway Net and DC-Bi LSTM,this article firstly proposes Highway-DC by integrating the two major architectures.Highway-DC uses the Highway algorithm in Highway Net to control the Dense Connection in DC-Bi LSTM,so as to control the propagation of information more flexibly.In addition,this paper further proposes an improved Highway algorithm,Highway II,which effectively solves the disadvantages of the Highway algorithm.The Highway II-DC based on it has excellent performance,and it shows greater advantages than Highway-DC and DC-Bi LSTM in accuracy,convergence speed,and maximum available network depth.2): Aiming at the problem of insufficient data,inspired by the hierarchical attributes of text and cropping,the famious data enhancement method,this paper proposes Hierarchical Data Augmentation(HDA).First,HDA uses the hierarchical attributes of text to enhance data from text and sentence levels.Secondly,at these two levels,HDA uses attention mechanisms to extract important part of words/sentences in the text,so as to "crop" the text.Experimental results show that HDA generates a large amount of high-quality data at both levels,which effectively improves the performance of the model.Compared with several existing data augmentation methods,the advantages of HDA are also more prominent.3): In order to improve the performance of the existing Hierarchical Attention Network(HAN)in obtaining the hierarchical attributes of text,this paper proposes Hierarchical Soft-Hard Attention Network(HSHAN).HSHAN uses a Soft Attention Mechanism and a Hard Attention Mechanism to improve the ability to obtain the word-level and sentence-level attributes of the text in the original HAN.The experimental results show that compared with HAN,HSHAN has a certain improvement on text classification tasks.