Research On Multi-Label Judicial Text Classification Algorithm Based On Attention Mechanism

The booming development of the Internet has driven tremendous changes in all walks of life,including the judicial field.The rapid growth in the number of judicial texts has prompted us to adopt more efficient techniques to classify them correctly,which contributes to the rational use of judicial resources.A judicial text can belong to multiple categories at the same time,which involves the problem of multi-label classification.Traditional methods tend to focus on documents to be classified,resulting in insufficient utilization of semantic information of labels and insufficient mining of label correlation.Meanwhile,they fail to take into account the different text content that different labels focus on.In response to the above problems,this thesis proposes a multi-label text classification model based on attention mechanism.By introducing the semantic information of labels,capturing the high-order correlation between labels,and combining the attention mechanism,it learns the implication for each label.The specific document representation of semantic information and correlation information realizes the interaction between label information and document information,so as to obtain richer and more comprehensive text features and use them for final classification.First,for judicial text and label semantic information,the advanced BERT pre-trained language model is used to obtain their respective word embedding representations.By representing them in the same vector space,the potential semantic relationship between judicial text and label semantic information is established.Compared with the static text representation method,the BERT model can obtain dynamic text representation with contextual information through the bidirectional Transformer encoder,which can effectively solve the problem of polysemy.Next,the label co-exist graph is constructed in the label correlation extraction module,and the global structure and local structure of the label co-exist graph are modeled using the structural deep network embedding method,thereby mining the high-order correlation between labels.Next,in the global context feature extraction layer,feature extraction is performed on the judicial text vector through the Bi GRU,and the extracted text features are input into the Label Semantic Information Attention（LSIA）module and the label correlation attention（LCA）module.Both LSIA and LCA adopt the attention mechanism to focus on the key information in the features,and capture the text features perceived by label semantic information and the text features perceived by label correlation respectively.Then,in the dual attention feature fusion layer,the features obtained by the LSIA module and the LCA module are fused by vector concat,and a specific text representation containing semantic information and correlated information is learned for each label,which preserves the maximum extent possible.Labels carry prior knowledge and establish the interaction between labels and documents.Finally,the fused features are input into the label output module,and the final multi-label classification is completed through the fully connected layer and the Sigmoid activation function.We carried out experiments on the Multi-CAIL2018 dataset,including: ablation experiments,feature fusion experiments,and compartive experiments to verify the overall performance of the model proposed in this thesis and the effectiveness of the LSIA module,LCA module,and feature fusion method.The experimental results show that the model proposed in this thesis achieves 94.32%,31.13% and 62.73% of the micro-average F1 value,macro-average F1 value and union F1 value,respectively,which are better than the benchmark model,which proves the effectiveness of the model proposed in this thesis.