Research On Dynamic Heterogeneous Interaction Graph Representation Learning

Graph modeling is the key technology for data analysis,which can describe complex connections and interactions among different nodes in real-world systems.Focusing on projecting high-dimensional sparse topological nodes into low-dimensional vector space besides keeping structural relevance,graph representation learning has become increasingly popular during the past years,for graph mining tasks such as node classification,link prediction and recommender systems.The current graph modeling and representation learning mainly deal with static homogeneous graphs,with the assumption that nodes and edges are of the same type and never change over time.However,in the realworld systems like social media,e-commerce platform,academic networks,and etc.,there are multiple types of entities connected with different-typed interactions,containing rich semantic information and complex dynamic evolving characteristics.For example,users' temporal interactions including clicks,collections as well as purchases not only reflect multiview shopping preferences but also showcase continuously changing demands.Most existing approaches cannot preserve dynamics and semantics on graphs,leading to the poor representations of nodes in realworld systems.Moreover,as the interactions continue to accumulate over time,the scales of such graphs become larger and larger.Traditional methods would suffer from expensive computational and memory cost.Therefore,representation learning modeling for dynamic,heterogeneous and large-scale graphical data is an urgent problem to be solved,which has very important theoretical value and broad application prospects.This thesis introduces the concept of dynamic heterogeneous interaction graph to model kinds of temporal interactions between different-typed entities in real-world scenarios,and focuses on studying the edge-level generation of heterogeneous interaction events,the semanticlevel dynamic semantic evolution,the integration of long-and short-term,and the acceleration strategies on large-scale heterogeneous interaction graph representation learning.To summarize,this thesis mainly investigates fundamental and vital technologies for dynamic heterogeneous interaction graph representation learning and its application as follows:First,aiming at modeling heterogeneous interaction event formation process,we propose the heterogeneous Hawkes Process based dynamic heterogeneous Graph Embedding approach(HPGE).By considering each current interaction as the result of continuing impact of multiple historical events,this model first designs the excitation measure to quantify the influence from heterogenous historical events,and then introduces heterogeneous Hawkes process to model the formation process of dynamic heterogeneous interaction graphs.In addition,to keep the efficiency,we design the temporal sampling strategy to extract representative events for learning.The effectiveness experiments and ablation study for node classification and temporal link prediction have verified that our HPGE outperforms most recent baselines obviously.Second,aiming at modeling dynamic semantic evolution,we propose the Dynamic Meta-path guided temporal heterogeneous Graph Neural Network(DyMGNN).This work is the first to introduce the concept of dynamic meta-path to search semantics with temporal bias.And then,it designs the corresponding sampling strategy to generate temporal neighborhoods for each semantic and the heterogenous mutual evolving attention to model the influence between different dynamic semantics,so as to construct dynamic semantic evolution of node representations.Experimental results for node classification and link prediction on three public datasets demonstrate the advantages of DyMGNN to state-of-theart alternatives as well as the effectiveness of the core designs including dynamic meta-path and mutual evolution modeling.Third,aiming at modeling the fusion of both long-and short-term heterogeneous sequential interest,we propose the Temporal Heterogeneous Interaction Graph Embedding(THIGE)which captures not only short-term evolving demands but also long-term habits to improve the quality of recommender systems.On the one hand,considering the dynamic and heterogeneous nature of user sequential behaviors,we design the recurrent neural network based short-term preference modeling to capture the current demands of users.On the other hand,since long-term historical interactions often imply multi-view shopping preferences of users and inherent qualities of items,we design the heterogeneous selfattention and habit-guided attention mechanisms to integrate long-and short-term preferences to construct representations of users and items for next item recommendation.Experimental results on three e-commerce recommendation datasets manifest that THIGE can achieve better performance and the designs are all effective for keeping advantages.Forth,aiming at accelerating large-scale heterogeneous interaction graph representation learning,we propose the heterogeneous importance sampling framework,including type-dependent and typefusion strategies,which can maintain model performance while significantly reducing memory and computational costs.To overcome the expensive time cost of node-wise sampling and the heavy memory resources of layer-wise sampling on homogeneous graphs,we design the batch-wise heterogeneous importance sampling strategies including typedependent and type-fusion samplers and the corresponding self-normalized and adaptive estimator to sample representative neighbors in both structure and semantic levels.We test the framework on five public datasets,and the experimental results demonstrate both the effectiveness and efficiency of our designs.The reductions in memory resources,time cost and computational edges are respectively up to 92.48%,85.95%,and 93.36%.