| In recent years,the development of artificial intelligence and deep learning technology has promoted the progress of human society in the fields of computer vision and natural language processing.However,computer vision and natural language processing are usually used to process the Euclidean data such as pictures,texts,and videos,and the processing method(graph neural network)for non-Euclidean data such as graph(Graph)has recently received extensive attention.Knowledge Distillation is a method of model compression.The purpose is to transfer knowledge from a complex and resource-intensive large model to a simple model through "distillation".The "distilled" small model has similar performance to the large model,so as to achieve the purpose of obtaining high performance with less resources.The work represented by regenerative neural network[1]is devoted to designing the distillation framework to make the performance of the student model surpass that of the teacher model.precision.Existing works still seldom use knowledge distillation to improve the performance and efficiency of graph neural networks,and few works design knowledge distillation algorithms for graph neural networks in different scenarios.This article aims to explore the characteristics and shortcomings of graph neural networks in different application scenarios,research and design knowledge distillation algorithms for graph neural networks,improve the deficiencies of graph neural networks in various scenarios,and talk about heterogeneous graphs and unsupervised two respectively.In order to solve the thorny problem of graph data mining and the important application problem of artificial intelligence-based protein structure prediction in biomolecules,a knowledge distillation algorithm for graph neural network is designed to improve the performance of graph neural network in these scenarios.Heterogeneous Graph is a graph with various types of nodes and edges.It has a wide range of applications in commodity recommendation,risk control and other scenarios.In this paper,we first study the characteristics and characteristics of graph neural networks on heterogeneous graphs.Insufficient,found two problems of the graph neural network in the heterogeneous graph scene,and proposed a new knowledge distillation framework HGPF,the experiment proved that in the semi-supervised node classification task,HGPF can effectively solve the heterogeneous graph neural network There are two problems,and the performance of HGPF surpasses all heterogeneous graph neural networks and model frameworks.Additional experiments(such as ablation experiments)also prove the effectiveness of our designed method and the rationality of motivation.Although HGPF has achieved excellent results on semi-supervised problems with heterogeneous graphs,in practical applications,supervisory signals are difficult to obtain and are usually noisy,so unsupervised learning on graphs is an important challenge,while traditional Unsupervised graph comparison learning algorithms usually use a fixed graph neural network for comparison.We have proved the shortcomings of this scheme and designed an unsupervised graph representation learning algorithm MA-GCL to solve this problem.Experiments show that MAGCL has achieved excellent performance in both node and graph-level tasks,and other experiments have also proved the effectiveness of teacher and student network design based on model enhancement,which shows that we can also design knowledge distillation algorithms in unsupervised scenarios Effectively make up for the shortcomings of the graph neural network.Finally,in order to study the applicability of the knowledge distillation algorithm for graph neural network,we selected the protein structure prediction problem,an important application in the interdisciplinary field of artificial intelligence&life sciences,as the target,and designed a protein structure prediction model based on graph neural network,and improve the performance of protein structure prediction through the method of knowledge distillation.The experiment proves that our method surpasses all baseline models and achieves the most accurate prediction effect on the task of antibody structure prediction,and we also designed an online system,available for users to query online protein structure prediction results. |
