Research On Contextual Image Labeling

Many fundamental tasks in computational vision can be formulated as predicting un-known properties of a scene from a static image. If the scene property is described by a set of discrete values in each image, then the corresponding vision task is an image labeling or understanding problem, which always suffers from great variations in inter-or intra-similarities of different classes. The main difficulties can be summarized into three aspects:1), Difficulty in modeling complicated visual patterns observed in natural images;2), Dif-ficulty in learning object category with most discriminative feature and model parameters;3), Difficulty in optimization for computing the final solution by given the learned model. This thesis takes a statistical modeling, learning and inference approach to the image un-derstanding problem, addressing the following main issuers in computer vision:1) how to efficiently encode highly structured and texture objects exhibited in nature images?2) what are the efficient representations of contexts to investigate the intersections among different categories for labeling?3) how do we learn image models and the context representations for a labeling task from data?4) how do we design a fast inference algorithm to obtain the global optimal labeling results that achieves object recognition and accuracy segmentation task?5) what is the relationship between image labeling and object recognition?The works of this dissertation concentrate on image labeling and scene understanding. Our dissertation unfold the research on the four aspects, as follows:(1) As usual, the low-level visual cues (e.g., color and texture) is able to be used for recognizing object categories. The shape information as middle-level visual feature, however, is a robust representation to describe structured objects (e.g., cow, horse and car). Embedding explicit shape model, such as deformable template, not only defines shape mask in guiding more precise segmentation, but also provides consistent labels for associated pixels. Furthermore, the contextual cues, such as co-occurrence, play important roles to improve the performance of image labeling. As a result, this dis-sertation proposes a dynamic and hybrid markov random filed model for producing a consistent labeling output, which seamlessly integrates low-, middle-and high-level visual cues. (2) As we all known, different objects are usually characterized with heterogeneous fea-tures. Different features have different discriminative ability to recognize object. This dissertation proposes an automatic learning algorithm to perform feature selec-tion according to the discrimination of each feature. By collecting marginal statistics of the features over the training examples, we start from an initialized reference mod-el to learn a sequence of probability models so as to minimize the Kullback-Leibler divergence (KLD) with respect to the underlying distribution.(3) This dissertation develops a data-driven composite sampling algorithm that integrates the bottom-up computation and top-down sampling for seeking the maximization of posterior probability under the data-driven Markov Chain Monte Carlo (DDMCM-C) paradigm. We proof the proposed inference algorithm converges to the optimal labeling output.(4) The contextual cues have multi-scale property. This dissertation proposes the concept of multi-scale contextual features, and embeds this feature to a linear integral model for the task of image labeling.The experimental results on nature scenes demonstrate the effectiveness and efficiency for image labeling.