Variational PDE-based image segmentation and inpainting with applications in computer graphics

This dissertation explores the applications of variational PDE models to image processing, computer vision, and computer graphics, and also to building efficient numerical algorithms and schemes. In particular, the areas of contributions are segmentation, inpainting, and matting. There are three separate topics in segmentation. The first is unsupervised segmentation, in which we propose and analyze a nonparametric region-based active contour model for segmenting cluttered scenes. The novelty is the use of the Wasserstein distance in segmentation, which is able to measure the dissimilarity between histograms, either continuous or discontinuous, in a reasonable manner. We employ a fast global minimization method to solve the proposed model. An advantage of this method is that initializations can be arbitrary to obtain a global minimizer. Moreover, our proposed model has several properties due to the use of the Wasserstein distance. A variant of the proposed model is presented to handle local illumination changes in an image.;The second topic of segmentation examines the issue of scale in modeling texture for the purpose of segmentation. We propose a scale descriptor for texture and an energy minimization model to find the intrinsic scale of a texture at each location. For each pixel, we use the intensity distribution of its local patch to determine the smallest size of the domain that can be used to generate neighboring patches. The obtained scale descriptor is applied for improving the segmentation model described above.;In the third topic of segmentation, we propose a multiphase segmentation algorithm based on Chan and Vese's two-phase piecewise constant segmentation model. The proposed algorithm recursively splits a partitioned region into two, starting from the largest scale, and automatically detects when all the regions cannot be partitioned further. The number of phases is not given prior and can be arbitrary, and the junctions of phase boundaries are implicitly dealt with. Additionally, the proposed model provides a full hierarchical representation of the structure of a image.;In the area of inpainting, we present a new technique that works well in both textured and non-textured areas of an image. Euler's elastica inpainting is a PDE-based variational model that works well for repairing smooth areas of an image while maintaining edge detail. However, it is slow due to a stiff, fourth order PDE and is difficult to control. On the other hand, texture synthesis techniques work well in inpainting for areas that contain repeating patterns. We combine these two techniques to accelerate and constrain the PDE solution. Instead of a stiff minimization, we have a combinatorial optimization problem that is quicker to solve.;In the area of matting, we propose a new algorithm that takes into account both texture and geometric structures of the foreground and background of the given image. We propose to utilize our inpainting algorithm for the matting problem, which extrapolates both geometric features and texture into unknown regions. The proposed matting algorithm improves previous algorithms, whose performance is uncertain in the presence of sharp discontinuities in the foreground and/or background.