Multi-label Text Classification Method Based On Hyperbolic Manifold Representation

Multi-label text classification is a fundamental task in natural language processing,targeting at classifying a piece of text into one or multiple labels.The main challenge of this task lies in the large number of labels,and the labels exhibit the unbalanced long-tail distribution,i.e.,the occurrence of the many tail labels is far less frequent than that of the few head labels.Although compared with traditional methods,neural networks can extract richer text features and improve the classification performance,they generally assume labels to be mutually disjoint,and the information contained in labels cannot be fully utilized,such as label correlations.Whereas labels usually have hierarchical structures,and text also contains intrinsic hierarchies,e.g.,syntax trees.Therefore,it can be challenging to learn mappings between text hierarchies and label hierarchies in multi-label classification.Considering that the tree-likeness of the hyperbolic manifold matches the complexity of symbolic data with hierarchical structures,the hyperbolic manifold representations present better inductive bias for text and label hierarchies.To make use of the representation capacity of the hyperbolic manifold for multi-label text classification,this thesis proposes to jointly learn the hyperbolic manifold representations of text and labels in the same hyperbolic space,which is specified in the following two parts:(1)Based on the Poincaré ball model of hyperbolic spaces,this thesis first introduces a Poincaré probe,which verifies that the hyperbolic manifold representations can capture richer hierarchical features from text.This thesis studies the contextual word representations of pre-trained language models on two probing tasks: syntax trees and sentiment analysis.The hyperbolic manifold representations of text obtained by the Poincaré probe can be intuitively visualized.In these two probing tasks,the Poincaréprobe better recovers the hierarchies of the text than the Euclidean probe.(2)In order to combine the text and label information,this thesis proposes a Hyperbolic Interaction Model(Hyper IM)for multi-label text classification.By learning the hyperbolic manifold representations of both text and labels,Hyper IM can take advantage of the internal hierarchical information.Hyper IM is designed to learn the labelaware text representations through fine-grained interaction between the hyperbolic manifold representations of text and labels,then makes predictions of the corresponding labels for the text.In addition,this thesis also proposes a partial interaction mechanism that improves the scalability of Hyper IM.Extensive experiments conducted on three benchmark datasets of multi-label text classification show that Hyper IM can realistically capture the structural relationship between text and labels,and further improve the performance for multi-label text classification compared with the state-of-the-art methods.