Research On Long Text Classification Algorithm Via Multi-model Fusion With Attention Mechanism

Text classification has always been a very valuable problem in the field of information processing.With the advent of the era of big data,this technology has also penetrated into all aspects of people's lives.In recent years,deep learning relies on the ability to understand text semantics to make great progress on short text classification problems,but the effect on long text tasks is not very satisfactory.Due to the rich number of words in long texts,many scenarios rely on traditional bag-of-words models and machine learning models for modeling have achieved good and stable results,especially those that rely more on keyword features(such as news topic classification).Therefore,this paper attempts to combine deep learning technology with traditional methods to improve the model's ability to understand long text semantics and thus improve classification performance.In this paper,through research on current technologies in the field of long text classification,it is found that the attention mechanism is very effective in solving the long dependency problem of long text tasks.The long text can be layered and the attention mechanism can be applied at each layer to Improve the model's ability to extract key semantic features at various levels of text.At the same time,compared to convolutional neural networks,the structure of recurrent neural networks is naturally suitable for understanding the semantics of long sequences.The use of variants such as GRU(gated recurrent unit)can also solve the problem of gradient explosion.However,due to the information overload of long texts,deep learning models still have their limitations when applied.To this end,this paper proposes to apply the sentence-level text semantic representation obtained from deep learning models to traditional machine learning models,and through tfidf based The bag-of-words model expresses the fusion of models,synthesizes the advantages of each model,and improves the performance on long text classification tasks.Finally,the experiments on the Fin Tech data set and the Daguan data set are performed on the model,which proves that the proposed model has advantages over the mainstream methods.In semantic understanding of long texts,attention mechanism and recurrent neural network can make it significantly improved,and combining sentence vectors with traditional machine learning models can also have a good effect.In addition,in the case where the base learner is excellent and sufficiently different,ensemble learning is sufficiently effective for long text classification tasks.Finally,the model proposed in this paper has good generalization performance.