Research Of Causality Based Representation Learning And Its Application

With the rapid development of deep learning and neural networks in recent decades,representation learning research,especially deep representation learning has received more and more attention.Related research which involves computer vision,natural language processing,and graph data analysis have achieved remarkable results.Although the current representation learning methods perform well in many tasks,with the deepening of research,it is found that the success of these methods is mainly due to the correlation relationship of the data,but too much dependence on the correlation inevitably makes the current representation learning have some problems: First,the traditional representation learning method is easily affected by the internal bias of the data.The indiscriminate use of biased correlations will cause the model to learn biased data representations.On the one hand,the representation cannot reflect the essential characteristics of objects; on the other hand,it is difficult for the model to make accurate predictions;second,traditional representation learning methods based on correlation often rely on the assumption of independent and identical distribution.If the distribution of the model in the training and testing data sets shifts,the correlation will become unstable.These unstable correlations will interfere with the predictive ability of the model,greatly reducing the robustness and generalization ability of the model.In order to solve the above problems,this thesis improves the representation learning process from the perspective of causality,learning unbiased representations with the help of causal inference methods on the one hand,and learning independent dynamic causal representations with the help of variational inference methods on the other hand,constructing a causality-based representation method,which in turn improves the prediction effect and generalization ability of the model.In terms of application,this thesis innovatively proposes two models,CausalRD and DCSVAE,which achieve performance improvements compared to their respective baselines in rumor detection and non-stationary temporal generalization tasks,respectively.As for the problem that it is easy to learn biased representations in traditional representation learning,the proposed CausalRD models the generation process which could lead biases in rumor dissemination through causal graph,and uses causal inference to simulate intervention operations in observational data,thereby eliminating popularity bias and conformity The effect of bias on user preference representations.In the prediction phase,the proposed model uses the graph neural network to capture the propagation structure of rumor events,and combines the text content for the rumor detection task.CausalRD is the first work which use causal inference to improve representation learning in the rumor detection task.The learned debiased representation can not only reflect more accurately user’s true preferences,but also bring better prediction results to the model in the rumor detection task.For the problem that traditional representation learning is overly dependent on correlation and leads to the decline of generalization performance,taking the generalization of non-stationary time series as an example,the DCSVAE model first analyzes the distribution deviation,proposes a causal diagram for non-stationary time series data and tries to model it The dynamic causal representation in it.In addition,this thesis also proposes constraints based on mutual information to ensure that the dynamic causal representation is fully disentangled from other latent variables.DCSVAE is the first work which learns dynamic causal representations in non-stationary time series generalization tasks.Experimental results show that the learned representations significantly improve the generalization ability of model.