Study On Bayesian Variable Selection For Finite Mixture Regression Model Based On Variational Inference

With the development of the information age,the data sets that we may collect may have the characteristics of heterogeneity and high dimension.When data comes from a heterogeneous population,it is usually not sufficient to fit a single model for the whole data set.It is necessary to identify the subgroups in the data and fit a model into each subgroup.Finite mixture regression models are often used to solve this kind of problem.In the case of high dimensions,we usually need to reduce the dimension of the data through variable selection,while the regression model of different subgroups may require different subsets of predictor variables to explain the response variables.Therefore,compared with the single regression model,the variable selection problem of the finite mixture regression model is more complex.We propose a Bayesian variable selection method to fit a finite mixture model of linear regression,which assumes that the data comes from a heterogeneous population consisting of several different subpopulations with different characteristics,within each subpopulation,the response variable can be explained by linear regression to the predictor variable.Based on the mixture regression model,we introduce spike-and-slab prior to identify the important predictive variables in each subgroup.At the same time,considering the same role of the same predictive variable in different sub-models,we extend the method and propose a group variable selection method for finite mixture regression models.In terms of algorithm,the traditional Bayesian variable selection method mostly adopts sampling methods such as MCMC algorithm,which has the disadvantages of long time consuming and occupying large computational resources in the case of big data.To this end,the VBEM algorithm based on variational inference is derived,which can realize the clustering of samples and the estimation of model coefficients simultaneously.We set up two groups of simulation experiments,and compared the two methods with the penalty likelihood-based method and the traditional Bayesian variable selection method,and verified the good performance of the two methods.At the same time,we also analyze the real data to further illustrate the effectiveness of the method.