A Model Selection Study For Finite Mixture Models

Statistical inference for finite mixture models with unknown number of components has long been challenging due to the issues of identifiability,degenerated Fisher matrix,and boundary parameters.In this paper,we consider the estimation of a mixture model that contains more components than the true one,as the information about an upper bound on the number of components is relatively easy to obtain.As far as we known,there are few studies about FMR with ultra-high dimensional covariates.It is of great interest to extend the ultra-high dimensional results in FMR,especially when the number of components is unknown.In chapter 2,a penalized likelihood method is proposed with a penalty on the difference of component parameters for mixtures with unknown number of components.We prove that the resulting estimator is in the non-degenerated area of the nonidentifiable subset of the parameter space,and show that the proposed new estimator could achieve the root-n pointwise convergence rate as the traditional maximum likelihood estimation.Further,we derived the modified EM algorithm to maximize the penalized log-likelihood function,and four simulated examples about Poisson mixtures and Normal mixtures are respectively conducted to demonstrate the finite sample performance of the estimation procedure.In chapter 3,we derive the condition for the penalty function to achieve a sparse estimation of mixing proportions.With a suitable penalty function on mixing proportions,the new estimator is proved to be consistent on the order selection,and have an asymptotic normal distribution.Conditions are provided to guide the selection for the penalty on mixing proportions.Our derived sparsity conditions also reveal some surprising but interesting differences among some commonly used penalty functions and explain why the performance of some popularly used penalty functions,such as Lasso and SCAD,provide unsatisfactory results in the order selection.An EM algorithm is provided and its ascent property is established for the proposed penalized likelihood method.Extensive simulations and a real data analysis are conducted to demonstrate the effectiveness of the newly proposed estimator.In chapter 4,we propose a penalized estimation method for finite mixture of ultra-high dimensional regression models.A two-step procedure is explored.Firstly,we conduct order selection with the number of components unknown.Then variable selection is applied to ultra-high dimensional regression models.A specific EM algorithm is designed to maximize penalized log-likelihood function.We demonstrate our method by numerical simulations which performs well.Further,an empirical study of return on equity(ROE)prediction is shown to consolidate our methodology.