Application Of Em Algorithm To Parameter Estimation Of Hierarchical Linear Models

The EM algorithm is one of the most useful methods for computing maximum likelihood estimates in models that can be formulated as missing data models. The main reasons for this are the fact that one can take advantage of the simplicity of complete-data maximum likelihood and of the nice convergence properties of the algorithm. In this paper, we consider to analyze and investigate the related problem about EM algorithm in the hierarchical linear models, mainly discussed as follows:1. Mixture models have been widely applied in the field of unsupervised statistical pattern recognition. However, the conventional estimation method can’t obtain the parameter estimation of mixture models. While the EM algorithm solves the problem well, we present this method by taking the normal mixture models which has two components for instance, and derived the iteration formula of EM algorithm of the parameters. Finally, the feasibility of this algorithm is verified by a numerical example.2. Hierarchical linear models can implement via the EM algorithm. However, because of its difficulty in calculating the explicit expression of the integral in E step, the application is limited. So this paper proposes a new algorithm, which is based on the MCEM algorithm, to implement the two layers hierarchical linear models. Thus the proposed algorithm has the advantages of EM algorithm and also has a wide range of applications. At last, the proposed algorithm’s effectiveness is illustrated by a numerical example. 3. The acceleration of MCEM algorithm, which is based on MCEM algorithm and N-R algorithm, improve the convergence rate of MCEM algorithm. However, while the model is complex, it is difficult to calculate the explicit expression of the integral in N-R step. Therefore, we propose an improved MCEM acceleration algorithm, which is to calculate the N-R step by the Monte Carlo simulation. Thus the algorithm has been successfully used in a wide range of applications. That is to say it facilitates N-R step by Monte Carlo simulation and also has quadratic convergence rate in a neighborhood of the posterior mode. Later its excellence in wide range of applications and convergence rate is illustrated by a classical example.