2.1D Sketch Of Images: Modeling,Inferring And Applications

2.1D sketch (layered image representation) is an important problem in the computer vision research to get depth information and recovery occlusion boundaries from a single image. It can be used in object recognition, motion analysis and tracking etc and it is an important issue in computer vision research.In this thesis we compute the 2.1D sketch from real images with a mixed Markov random field in a Bayesian framework. According to speciality of 2.1D sketch, it's need to build up a suitable representation for the problem firstly, then still there are three important tasks to be handled simultaneously: region coloring/layering, contour completion and ambiguity maintenance.In order to resolve the 2.1D sketch problem, a three-layer image model from the image to the 2D representation to the 2.1D representation is built up firstly, then focuses on how to model and infer a 2.1D sketch from the 2D representation.Given an input image, its 2D representation is computed firstly, which includes a set of 2D segmented regions and a set of detected 3-degree-junctions such as T-junctions, Y-junctions or arrow junctions, which will be broken into terminators dynamically during the procedure of region layering. The terminators are assigned ownership to the regions as open bonds (uncompleted) or so called address variables. The closed contours are from the primal sketh of input image. In the process of inferring, terminators will be assigned address variables by Gibbs sampling in each layer and contours will be completed by Elastica method.Through a stochastic graphical representation, the 2D representation builds up a reconfigurable graph using a mixed Markov random field, which consists of two types of vertices: (1) regions, acting as the nodes traditionally found in graphical models, such as Markov random fields or Bayes nets (2) address variables, acting as a new kind of node for the mixed MRF.There are regions set, terminators set, surfaces set and contours set in the 2.1D representation. The regions set include layer number and label information for each layer, terminators set include the label information of each layer, surfaces set include surfaces number and included regions and contours set include the completion informations, such as inserted curve.To move from 2D to 2.1D, we present an inference algorithm that maximizes a joint Bayesian posterior probability with two terms: (1) region coloring/layering based on Swendsen-Wang Cuts algorithm for the partitioning of a region adjacency graph to obtain the partial occluding order relations; (2) address variable assignments based on Gibbs sampling for the completion of open bonds. After the inference is complete, the regions are arranged into a layered representation and we are able to obtain some previously occluded surfaces that are merged from regions by contour completion.To keep the intrinsic ambiguity of the 2.1D sketch problem, multiple distinct solutions are obtained by sampling the Bayesian posterior probability with K-adventure. We perform experiments on a variety of images and satisfactory results are obtained.After inference is finished, regions are merged into some surfaces and then are assigned into several layers, according to the layered representation, address variables are assigned to corresponding terminator, then in each layer, contours can been completed with Elastica method which inserts a smoothed curve by computing position and orientation information of two termators.As a special issue of the 2.1D sketch, a cartoon generation method is introduced in this thesis. Firstly computing the sketch parts and the segmentation parts of the image by utilizing interactive methods, then inferring previous results into layered vector graph, a vector cartoon picture generated, each parts of the picture can be edited or reorganized into another object by users. Many images are tested in the thesis and the experimental results show that the cartoon generation method can work well.