EM Algorithm And Its Application Research Based On Gaussian Mixture Model

Gaussian mixture models(GMM) has been widely applied in diverse fields as pattern recognition, computer vision, machine learning, data mining, and bioinformatics, to complete diverse tasks as image segmentation, clustering, and function fitting. Expectation-Maximization(EM) algorithm is usually used to solve the parameters in GMM. Although EM algorithm is very effective and can guarantee the convergence, it has two open problems:(i) it is sensitive to initialization,since EM can only guarantee to converge to the local optimum;(ii) the number of Gaussian components in GMM needs to be specified in advance, whereas this number is usually hard to set without any prior information.The contribution of this paper mainly contains two parts:In the 1st part, we seek to solve the problems that EM algorithm has. This paper analyzed and pointed out that the initialization sensitivity problem for EM is largely due to that the competitive relationships among all Gaussian components, involved in the parallel learning process of EM algorithm, leads to the demanding requirement for the fair initial competition condition. Accordingly, from the perspective of strategy, a serial learning process, similar to EM algorithm, is added ahead of EM algorithm, that is, the competition should be avoided before the fair competition condition is obtained. In this way, influence of random initialization to EM can be largely reduced. In the serial learning phase, all Gaussian components try to fit with their own clusters, so as to create a fair competition condition; in the parallel learning phase, EM is used to fine-tune the above serial learning result, so as to reach a global optimum. Moreover, the number of Gaussian components in GMM doesn’t need to be specified in advance. Based on the proposed method, GMM model was then applied to cluster analysis. The result demonstrated that the proposed method largely improves the clustering performance of EM.In the 2nd part, we seek to inherit the merit of EM, in order to provide a more general learning model to help people solve problems in diverse fields. Firstly, this paper pointed out that EM is in nature a heuristic learning process, and extracted from EM a general heuristic learning model. In order to demonstrate its effectiveness, this learning mode was then applied to several specific tasks as tracking, perceptual organization and contour detection, which to some degree demonstrated this learning model can be a general unsupervised learning model and can be of general meaning.