| It's a scientific dream that we could build machines which understand the contents of natural images. The realization of this dream is challenged by the enormous complexity of natural images. The difficulty lies in two major aspects: (1) the modeling problem-how to model abundant patterns that emerge in images such as texture, lighting, shape, motion etc. (2) the computing problem-how to make inference of these patterns. Although the modeling problem has become increasingly clear by a number of recent developments [6, 49, 61, 65, 93, 96, 98], etc., designing a general and efficient computing framework remains a big challenge. Previous attempts in designing such a framework failed either because they are not general enough to deal with problems of high complexity or due to efficiency problem. We attack the computing problem by presenting a new paradigm called Data-Driven Markov Chain Monte Carlo (DDMCMC) in which exploration of the heterogeneous solution space is guided by importance proposals probabilistically in a efficient and robust manner. We make the contributions as follows: (1) We propose a stochastic computing paradigm called DDMCMC for visual inference and summarize many traditional vision problems for example, image segmentation, object recognition, perceptual organization etc., into a general problem called image parsing. It decomposes images into their natural patterns such as texture regions, curves, faces, etc. (2) In our first attempt to build an image parsing system, we use the DDMCMC paradigm and device a novel algorithm called image segmentation by DDMCMC wherein patterns of interest are regions of unknown parameters from seven known types. This algorithm has been tested extensively for a large set of images. (3) We then introduce two generative curve models to extend the system to parse images into regions and curves. (4) Two high level patterns of structure, parallel curves and trees curves, are further incorporated into the system so that procedures of perceptual organization are carried out in an early stage of image parsing. (5) At the end of this thesis, we also provide some proofs of how to achieve efficient MCMC algorithms. |
