Research On Correct Convergence Of The EM Algorithm For Gaussian Mixtures

The expectation-maximization(EM) algorithm proposed by Dempster, Laird and Rubin in 1977 is a general methodology for maximum likelihood(ML) or maximum a posterriori(MAP) estimation when the observations can be viewed as incomplete data. Since EM algorithm has certain good convergence properties and can be widely applied, it has attracted many researchers from statistics, other applied science and engineering. The distribution of mixture of densities is very popular in practice and the data subject to such a distribution can be viewed as a kind of incomplete data. So, the EM algorithm for mixtures of densities, particularly for Gaussian mixtures, is very important. Recently, the progress has been made on the research of the EM algorithm for Gaussian mixtures. Actually, it has been found that the convergence rate of the EM algorithm for Gaussian mixtures is related to the overlap measure of Gaussians in the mixture in certain situation. As the overlap measure of Gaussians in the mixture tends to zero, the EM algorithm tends to be asymptotically su-perlinear. That is, the convergence speed of the EM algorithm becomes much fast as the overlap measure of Gaussians in the mixture tends to zero, which conforms to simulation results and practical applications. However, there has been still an important but unsolved problem: whether the EM algorithm can converge to the correct solution. It follows from the general convergence theory that the EM algorithm generally converge to a local maximum solution of the likelihood function and cannot be guaranteed to converge to a correct solution, i.e., a consistent solution of the samples. But in practical applications and experiments, we often find that EM algorithm for Gaussian mixtures can converge correctly when the overlap measure of component densities in the mixtures becomes small.In this paper, we present a theoretical analysis on the correct convergence of the EM algorithm for Gaussian mixtures. We first introduce the EM algorithm and its convergence properties. We also give a variation of the EM algorithm forGaussian mixtures. We then prove that the EM algorithm becomes a compact map in certain neighborhood of a consistent solution when a measure of the average overlap of Gaussians in the mixtures is small enough and the sample size is large enough. We further obtain and prove the condition of the correct convergence of the EM algorithm for Gaussian mixtures. Finally, the theoretical results have been demonstrated by the simulation results.