| It is still an urgent problem to fully and efficiently learn the structure information and attribute information(features)required for graph convolution networks,and to effectively integrate them into graph neural networks is still an urgent problem to be solved.To solve this problem,there will be many challenges,such as graph construction,over fitting,model computing efficiency,over smoothing,model optimization,and embedded propagation.Therefore,this thesis focuses on the challenges of computational efficiency,over smoothing phenomena,model optimization,and embedded propagation in semi-supervised learning based on graph neural networks.The main works are as follows:1)Existing graph neural networks usually rely on a specific graph convolution which has high computational complexity.To reduce the complexity,this thesis propose a novel method called Semi-supervised learning based on Multiple Layer Graph Regularized Perception(MLGP).The specific method is to design a multi-order neighboring contrastive loss and adopt gated mechanism,which implicitly uses multi-order graph topology information and captures important features of nodes.More discriminant features can be obtained.MLGP is more effective and more robust than existing methods when dealing with large-scale graph data with noisy adjacency information,which is also confirmed in contrast experiments.2)Existing graph attention networks only employ the first-order attention mechanism and thus fail to fully exploit and learn node’s contextual feature representations.To address this issue,this thesis propose Graph Context-Attention Network(GCAN)based on low and high order aggregation.The proposed model fully exploits both low-order and high-order information of nodes for representation learning by employing high-order attention mechanism and adversary regularization constraints.The main benefit of the proposed method is that it can effectively capture the discriminative feature differences among node representations while enhancing the diversity and richness of node features.Therefore,it can alleviate the issue of over-smoothing in the network learning.We perform extensive semi-supervised node-level analysis experiments on nine datasets and the experimental results demonstrate the effectiveness and better performance of our approach.3)Existing graph neural networks generally either employ one classifier head which usually makes the model fall into local minima or multiple classifier heads independently regardless of the correlation between them.To overcome this issue,in this thesis propose Graph Neural Networks based on Multi-head Collaborative Learning(MCL-GNNs).The MCL-GNNs are designed with multiple classifiers to implement collaborative learning processes via semi-supervised learning optimization algorithms used to complement each other’s information.Experimental results on several benchmark datasets show that our method has better generality and stability than the other methods.4)Existing multi-head self-attention neural mechanisms generally integrate the representations of multi-heads via some simple fusion strategies,such as concatenation,summation,mean without fully considering the dependence and interactions of multi-heads.For this deficiency,we design a multi-head transformer which is applied into the head dimension to explore the relationships of different heads for multi-head representations.This thesis integrate the proposed multi-head transformer module into Graph Attention Network architecture and propose a novel graph neural network for graph data representation and learning,called Graph Attention Networks based on Multi-head Transformer(Head Former).By this means,the proposed method enables the collaboration of different heads and enhances the representations of multi-heads.The effectiveness of the proposed method is verified with semi-supervised node-level tasks on several datasets. |
