Graph Representation Learning For Multi--modal Data

The continuous developments of the internet and digital multimedia technology have re-sulted in the explosive growth of data.The data form has also developed from a single text to multi-modal data including text,images,videos,and so on.In recent years,deep learning technology has provided a strong guarantee for multi-modal big data analysis and achieved excellent results in many fields.However,traditional deep networks cannot handle the relationship between samples,resulting in low data utilization.Moreover,existing deep learning methods rely on a large number of labeled samples which will consume huge label costs.How to solve the dependence of deep models on labels and improve the efficiency of data usage have become an important issue.Graph representation learning can represent data as nodes and their relationships.By embedding these nodes and relationships to a low-dimensional space,the semantic information between data can be captured,which can be applied to different data mining tasks flexibly.Thus,the construction of graph representa-tion models and the study of graph representation learning methods for multimodal data have become a new research hotspot in the field of data mining and representation.The graph representation learning method first constructs graph data based on samples and adopts graph data as supervision to further optimize the model.Hence,constructing a high-quality structure graph is a prerequisite for graph representation learning to achieve good results.In addition,extracting discriminative information from graph data is the key to achieving good recognition accuracy.This thesis is dedicated to the study of multi-modal data graph representation learning methods which aim at the problems of existing methods,such as low feature representation ability,weak sample relationship,insufficient semantic mining.poor model robustness.We innovated theoretical methods from the construction of graph data and optimization of graph structure and explored new applications of graph models.The main innovations of this paper can be summarized as follows.A novel dual autoencoder network for deep graph clustering is proposed.Almost all exist-ing deep clustering methods endeavor to minimize the reconstruction loss.The hope is to make the latent representations more discriminative which directly determines the clustering quality.However,in fact,the discriminative ability of the latent representations has no sub-stantial connection with the reconstruction loss,causing the performance gap of the noisy graph matrix.Based on it,a dual autoencoder,which enforces the reconstruction constraint for the latent representations and their noisy versions,is utilized to establish the relationships between the inputs and their latent representations.Such a mechanism is performed to make the latent representations more robust.In addition,we adopt the mutual information estima-tion to reserve discriminative information from the inputs to an extreme.In this way,the decoder can be viewed as a discriminator to determine whether the latent representations are discriminative.Furthermore,deep graph clustering is harnessed to embed the latent repre-sentations into the eigenspace,which followed by clustering.This procedure can exploit the relationships between the data points effectively and obtain optimal results.The proposed dual autoencoder network and deep graph clustering network are jointly optimized.Empiri-cal experiments demonstrate that our method outperforms state-of-the-art methods over the five benchmark datasets,including both traditional and deep network-based models.An adversarial learning algorithm is proposed to improve the network robustness in deep graph clustering.The majority of existing deep clustering methods endeavor to minimize the reconstruction loss,and their goal is to make the target embedding space more discrim-inative since the embedding space directly determines the clustering quality.However,the embedded features are extremely susceptible to a small perturbation and lead to disparate clustering results.We attempt to define an adversarial sample of the embedding space,which easily fools the clustering layers but does not impact the performance of the deep embedding.Meanwhile,we present a powerful adversarial attack algorithm to learn a small perturba-tion from the embedding space against the clustering network.In this way,those unstable samples that are very likely to yield diverse clustering results are explored explicitly.More-over,we provide a simple yet efficient defense algorithm to optimize the clustering network,which can alleviate the differences caused by the perturbation.The experimental results on four popular datasets show that the proposed adversarial learning algorithm can optimize the feature distribution to alleviate the effect caused by a perturbation,therefore enhancing the robustness of the existing clustering framework.A new relaxation for the multi-way graph cut clustering method is proposed to obtain much clearer cluster structures.Exploring manifold information in multi-way graph cut clustering,such as ratio cut clustering,has shown its promising performance.However,the traditional multi-way ratio cut clustering method is NP-hard and thus the spectral solution may deviate from the optimal one.In this paper,we propose a new relaxed multi-way graph cut cluster-ing method,where?_2,1-norm distance instead of squared distance is utilized to preserve the solution having much clearer cluster structures.Furthermore,the resulting solution is con-strained with normalization to obtain a more sparse representation,which can encourage the solution to contain more discrete values with many zeros.For the objective function,it is very difficult to optimize due to minimizing the ratio of two non-smooth items.To address this problem,we transform the objective function into a quadratic problem on the Stiefel mani-fold and introduce a novel yet efficient iterative algorithm to solve it.Experimental results on several multi-modal benchmark datasets show that our method significantly outperforms several state-of-the-art clustering approaches.A self-supervised semantic alignment optimization method for graph neural networks is pro-posed.Graph neural networks(GNNs)are a powerful deep learning approach and have been successfully applied to representation learning on graphs in a variety of real-world appli-cations.Despite their success,two fundamental weaknesses of GNNs limit their ability to represent graph-structured data:poor performance when labeled data are severely scarce and indistinguishable features when more layers are stacked.We propose a simple yet effective Semantic Alignment Graph Convolution Network,which consists of two crux techniques:Identity Aggregation and Semantic Alignment,to overcome these weaknesses.The basic idea is the node features in the same class but learned from semantic and graph structural as-pects respectively,are expected to be mapped nearby.Specifically,the Identity Aggregation is applied to extract semantic features from labeled nodes,the Semantic Alignment is utilized to align node features obtained from different aspects using the class central similarity.In this way,the over-smoothing phenomenon is alleviated,while the similarities between the unlabeled features and labeled ones from the same class are enhanced.Experimental results on five popular datasets show that the proposed methods outperform state-of-the-art methods on various classification tasks.A heterogeneous graph neural network for multiple target domain adaptation is proposed.Domain adaptation,which transfers the knowledge from a label-rich source domain to un-labeled target domains,is a challenging task in machine learning.The prior domain adap-tation methods focus on pairwise adaptation assumption with a single source and a single target domain,while little work concerns the scenario of one source domain and multiple target domains.Applying pairwise adaptation methods to this setting may be suboptimal,as they fail to consider the semantic association among multiple target domains.We propose a deep semantic information propagation approach in the novel context of multiple unlabeled target domains and one labeled source domain.Our model aims to learn a unified subspace common for all domains with a heterogeneous graph attention network,where the transduc-tive ability of the graph attention network can conduct semantic propagation of the related samples among multiple domains.In particular,the attention mechanism is applied to op-timize the relationships of multiple domain samples for better semantic transfer.Then,the pseudo labels of the target domains predicted by the graph attention network are utilized to learn domain-invariant representations by aligning labeled source centroid and pseudo-labeled target centroid.We test our approach on four challenging public datasets,and it outperforms several popular domain adaptation methods.