Using Minimax Entropy Gibbs Model To Learn The Distribution Of Structural Gibbs Point Process And Random Vector

Gibbs distribution is often accepted as a general form of statistical models. Gibbs models, which bear this from, have been widely studied and applied in the field of image processing, computer vision and statistical learning. This thesis presents an in-depth work in the study of Gibbs models based on the Minimax Entropy Principle, which leads to some improved methods in modeling and training in the distribution learning of structural point process and random vectors.The thesis is based mostly on three recent papers on learning gibbs distributions. They are [Zhu 97]'s work on creating a unified statistical framework by minimax entropy principle, which is then used to learn texture images, [Guo 01]'s proposal for learning the distribution of intrinsic texture element (Textons) by gibbs model and [Liu 01]'s introduction of the minimax Entropy idea to the learning of Inhomogeneous Gibbs Model (IGM). These three papers study and develop, for different fields and practical problems, the gibbs models based on the minimax entropy principle. [Zhu 97] uses the principle to construct the statistical model for texture image and results in a distribution function that has the gibbs form. Thus a whole framework for analysis and synthesis of textures is proposed and achieves good results. On the other hand, [Guo 01] uses it to study how textons are laid on texture images. The texton map which conveys this layout information is a kind of structural random point process. The number, positions and attributes of textons are the objects that the model has to handle. While the above two papers mainly focus on how to model the spatial relations of variables on a field (Spatial Statistics), IGM targets at learning the distribution of data variables that are independently and identically distributed. IGM is generally more powerful than the Principle Component Analysis (PCA) in describing complex distributions; it is also much easier to estimate than the Gaussian Mixture Model (GMM). IGM provides a rigorous way for many learning tasks.This thesis contributes in the following two ways: First, in recognizing that the key element for a successful random field or point process model is to define the suitable clique space, and use it as an inter-medium for comparing two arbitrary random fields or point processes, we propose a robust and straight forward model to study the distribution of textons. The new model establishes the clique set by Delaunay triangulation, and requires that the synthesized texton map have the same statistics over these cliques as that from the observed map. We also adopt an automatic feature extracting strategy that is able to increasingly add effective features to the model. Experiment demonstrates that we are able to learn the distribution of textons with fewer features. Second, we also find that there are two drawbacks of the present IGM. It lacks a reasonable and measurable way to evaluate the effectiveness of each feature, and it takes extremely long time to train the model even for a very simple task. We propose in this thesis an information gain based principle to evaluate each candidate feature, requiring that the new feature with the maximum information be accepted. Simultaneously, an analytical form solution for the model parameters could be obtained. The model constructed in this way is very close to the final optimized one, thus Importance Sampling method is used to speed the estimation process. Experiments demonstrate that the improved training method for IGM achieves a speed about a hundred times faster.