Research On Deep Graph Representation Learning

As a type of relational data,graphs which are consisted of nodes and edges can model a wide range of complex data and systems in the real world.For the current deep graph representation learning research,one main problem is how to extract rich semantic and structural information in graphs.Although graph neural networks,which own strong expressive power and generalization capability,have achieved great progress,there are still many limitations and challenges that have not yet been resolved.ⅰ)Due to the irregularity and complexity of the graph structure,most of the existing graph neural networks use simple average/summation/weighted summation operation to aggregate the neighborhood node features,and then a single nonlinear transformation is applied to the aggregated features to update the central node representation.It collapses the features of all nodes in the neighborhood to the central node,leading to the loss of the semantic as well as local structural information.ⅱ)The message passing mechanism essentially makes the features of adjacent nodes in the graph propagate mutually.As the number of network layers increases,the aggregated features will gradually accumulate and be transferred to farther nodes,then all nodes in the connected subgraph will tend to have similar representations,which seriously reduces the discriminability between nodes,and results in over-smoothing.ⅲ)Many methods extract supervisory signals from the data itself to construct objective function and learn node representations in an unsupervised manner.Among them,methods that are constructed upon the mutual information maximization objective only construct mutual information constraints in the feature space,and lack effective modeling of the topological mutual information,leading to the deficiency of graph structure relationship maintaining as well as model encoding capabilities.To resolve the above problems,this paper studies the graph neural network model to enhance the expressive power of the model,capture fine-grained semantic as well as structural information,and improve the discriminability between nodes.The main research contents of this paper are as follows:(1)Aiming at the problem of node signal entanglement of the traditional average/summation aggregation methods,which is prone to cause over-smoothing and distinguishing difficulties between the node features,a hashing graph representation learning method is proposed.The irregular neighborhoods are first projected into a regularized bucket space,then node features in each hash bucket are aggregated,and different nonlinear transformations are imposed on different hash buckets.In addition,the positive pointwise mutual information matrix(PPMI)is used as a graph structure to capture the global semantic information,and the local and global prediction consistency constraint is constructed.The proposed method achieves better accuracies on publicly available node classification datasets,verifying the effectiveness of this method.(2)To resolve the problem that isotropic graph convolution method is easy to lose structural and semantic information,a graph deformer network(GDN)for graph representation learning is proposed.GDN first defines and generates multiple anchors,and projects node features to anchors,so that the irregular local neighborhoods could be transformed into regular anchor space.Nonlinear transformations are then applied onto different anchor directions,which could encode fine-grained node representations.In addition,the relationship between the expressive power of GDN and the Weisfeiler-Lehman graph isomorphism test is theoretically analyzed.GDN achieves better or comparable performance on public node and graph classification datasets,and its effectiveness is verified both theoretically and experimentally.(3)Targeting at the problem of insufficient node feature representations of the traditional average/summation graph convolution method which is prone to cause information loss,a Gaussian mixture encoded graph representation learning(GMGC)method is proposed to model the distribution of local neighborhoods and implicitly build different orientations for the irregular local neighborhoods.Further,in order to depict directions of variations,GMGC encodes the local neighborhood as Fisher Vector.To better characterize contextual information,GMGC explores a neighborhood construction method based on random walk,and samples multiple paths to comprehensively preserve local topology and capture long-distance node interactions.Moreover,in order to reduce the computational burden,a Gaussian mixture encoded graph pooling operation is introduced to reduce the graph size layer by layer.Extensive experiments verify the effectiveness of GMGC on multiple graph classification datasets.(4)In an attempt to address the problem that current mutual information based graph representation learning methods lack the mutual information modeling of graph structure,a deep graph structural infomax(DGI)model is proposed from the perspective of information bottleneck,which explicitly derives the structural mutual information constraints to guide the network learning in a self-supervised manner.Essentially,DGI maximizes the structural mutual information both edge-wise and local neighborhood-wise,while maximizing local as well as global representational mutual information.Further,a unified framework is developed to integrate structural and representational mutual information constraints as graph mutual information constraints.Experimental results on multiple node classification datasets verify the effectiveness of the proposed method.