Research On Causal Effect Estimation Model Based On Knowledge Distillation

Causal effect estimation refers to excluding the influence of confounding factors from massive and complex data,and estimating the impact of an treatment on individuals.It has been widely applied in practical scenarios such as marketing and biomedical science,providing assistance for people’s decision-making in production and life.The prevailing causal effect estimation models can be divided into incremental models and response models according to the output type.The former directly predicts the increment,which is usually realized by the"decision tree-like" model.The latter predicts the results of treatment and non-treatment respectively,which is usually realized by neural networks.The former has good interpretability,a small number of parameters,and a fast running speed.However,due to the influence of hyper-parameters,the model training is relatively unstable.The latter can receive a large number of features and have great flexibility in model structure,but it is also prone to over-fitting.A causal effect estimation framework based on knowledge distillation and sample matching called KDSM is creatively proposed in this thesis which combines the advantages of the two kinds of models.The main contributions of this thesis are as follows:Firstly,under the theoretical framework of knowledge distillation,a novel mode of information transmission is proposed,where the incremental model serves as the teacher model,the response model serves as the student model,and the incremental knowledge is treated as the extra learning goal of the student model.Secondly,to combine the two types of models and train them with real samples,this thesis also proposes a method of sample matching in the leaf nodes of the tree model.Under the circumstances of random experiments,random matching before each epoch is adopted in the leaf node,while in observational experiments,the sample matching strategy is nearest neighbor matching based on the propensity score.Through constructing the pairwise loss function,the relationship of the two relevant tasks is established between two different models,driving the transfer of information between models.Finally,abundant experiments have proved the correctness,effectiveness,and universality of the framework,which can improve the performance of the student model.In the random experiments,compared with non-distillation,the Qini coefficient on the Criteo dataset increases by 16.9%,and the Qini coefficient on the Production dataset increases by 19.6%.Compared with other algorithms,the Qini coefficient increases by 13%on the criteo dataset;On the production data,the Qini coefficient increases by 15%.In the online experiments,the KDSM group maintained an roomnight increment of 99.7%compared with the group of sending coupons to all users,and the cost was only 73.5%.Compared with nondistillation,the cost of each incremental roomnight is reduced by 6.5%.In the non-randomized experiment,FlexTENet+UpliftDT(ED)can reduce the PEHE by 17%on the IHDP dataset.In summary,this topic proposes a causal effect estimation model based on knowledge distillation and sample matching under the idea of multi-objective learning.It has taken into account two scenarios:random experiment and non-randomized experiment,abundant experiments have been conducted to prove its effectiveness and universality.