Network Representation Learning With Local Semantic Pattern Mining And Prompting

The research on the basic theory and frontier technology of big data and artificial intelligence has been attached great importance by the Chinese government and taken into consideration at the national strategic level.However,due to the rich correlation and implicit network mode contained in heterogeneous network,it is difficult to obtain the required information by accessing all data.How to efficiently acquire knowledge from massive and complex data and get rid of "information overload" and "resource misdirection" to provide expressive network representation for specific downstream tasks,which has prospective research significance and high application value to relieve the pressure of presentation learning process in practical application tasks.Despite the promising representation learning of graph neural networks(GNNs),the supervised training of GNNs notoriously requires large amounts of labeled data from each application.An effective solution is to apply the transfer learning in graph:using easily accessible information to pre-train GNNs,and fine-tuning them to optimize the downstream task with only a few labels.Recently,many efforts have been paid to design the self-supervised pretext tasks,and encode the universal graph knowledge among the various applications.However,they rarely notice the inherent training objective gap between the pretext and downstream tasks.This significant gap often requires costly fine-tuning for adapting the pre-trained model to downstream problem,which prevents the efficient elicitation of pre-trained knowledge and then results in poor results.Even worse,the naive pre-training strategy usually deteriorates the downstream task,and damages the reliability of transfer learning in graph data.To bridge the task gap,we propose a novel transfer learning paradigm to generalize GNNs,namely graph pre-training and prompt tuning(GPPT).Specifically,we first adopt the masked edge prediction,the most simplest and popular pretext task,to pre-train GNNs.Based on the pre-trained model,we propose the graph prompting function to modify the standalone node into a token pair,and reformulate the downstream node classification looking the same as edge prediction.The token pair is consisted of candidate label class and node entity.Therefore,the pre-trained GNNs could be applied without tedious fine-tuning to evaluate the linking probability of token pair,and produce the node classification decision.Moreover,existing graphrepresentation learning model predominantly emphasize the original noisy topological structure,but rarely notice the generalizable implicit connection in the local patterns,which prevents the efficient elicitation of the structure and semantic knowledge.To this end,we consider the powerful expressive ability of the prompt and propose a structural-aware Motif-based prompt tuning for graph clustering framework,namely MPTGC.Specifically,we first leverage motif-based prompt to match important local patterns and augment the structure and semantic information of original input graph to reduce the impact of the long-tailed distribution and noise in graph structure.Then,we utilize pretext tasks to pre-train a transferable backbone that can quickly adapt to graph clustering tasks.Based on the pre-trained model,we leverage task token prompt function to reformulate the objective of the downstream graph clustering task to look more like general feature reconstruction.Therefore,the pre-trained model could be directly applied without tedious fine-tuning to obtain the prediction probability.The extensive experiments on benchmark datasets demonstrate the superiority of our method.