On Probability Mixture Model And Applications

Probability mixture model is very popular in density estimation and clustering.However, its plain form is usually incompetent for some specific applications, suchas adaptive learning, large-scale classification, and multi-task learning. In this article,we will investigate its extensions in terms of the aforementioned three aspects.First, a recursive mixture model based on a "Generic to Specific" learning strategyis proposed. It starts as a "generic model" learned off-line, then "actively" detectsthe potential positive samples in a specific domain to update itself, and finally evolvesinto a "specific model" for that domain. It is applied to learning adaptive skin modelunder different illumination conditions. Comparing to the traditional methods, the skinregions detected by our method is more accurate.Second, a discriminative mixture model, named support cluster machine(SCM),is proposed. SCM combines the advantages of both Bayes optimal and max-marginclassifiers: 1) it adopts Gaussian mixture models as the training units to reduce thesample size as well as retain the original data distribution; 2) it maximizes the marginbetween positive and negative clusters to improve the generalization ability. SCM cansignificantly reduce the time complexity in large-scale classification.Third, a two-sided mixture model, named rating-matrix generative model(RMGM),is proposed. By co-clustering the rating matrices from multiple related domains, theusers and items in each rating matrix can be viewed as drawing from the same RMGM,which thus becomes a bridge among multiple domains for knowledge transferring andsharing. The experimental results validate that RMGM indeed can gain additional usefulknowledge from other domains for a certain domain.The proposed three novel extensions for probability mixture model are used forsolving three classical machine learning problems. Comparing to the existing methods,they can not only obtain better results in the empirical tests, but also provide promisingsolutions for solving these problems.