Research On Image & Video Matting

With the development of digitized acquisition technology, the prevalence of contemporary digitized appliance, the rise of media industry, the bloom of movie field, and the extensive application of VR (Virtual Reality) and AR (Augment Reality), the tie between graphics research and life is becoming more and more tighter and closer. The fundamental goal of graphics research is to represent the real world, and the preliminary requirements to representation are the faithful narration, easy manipulation, quick response. However, there exists such complex objects as hair, shadow and tree to difficulty depict, how to represent them are the current hot. Digital matting focuses on extracting the interested objects from image/video, and compose them into the novel context. Matting could manipulate the complex geometry object equivalently with 2D operational manner. The thesis concentrates on the matting and its key issues.The dissertation investigates four problems covering image matting, video matting, shadow extraction and image composition.Hard matting and soft matting are the generally so-called image matting. Graph Cuts and Poisson matting are the representative method of hard and soft matting sep-aratively. Subject to the influence of outlier among user interactions, matting based upon Graph Cuts requires additional modifications. In thesis, according to the second smallest eigenvector after image is transformed with locality preserving projections, the image is partitioned into the connected and independent regions. Different types of region require different strokes, hence the user interaction is guided and the user interaction independent of the final result. Poisson matting assumes that foreground and background are smooth, and take the gradient as the matting features, but it require the additional local operations to refine the result. an extension to poisson matting is given, which maintains the tensor in the matte, and could achieve the high-quality result.Space-time video volume simultaneously exhibits the spatial and temporal properties, and it can be exploited for assisting the video editing. Due to the huge number of graph node, the conventional video object extraction method based on Graph Cuts shows the low efficiency and has slow response to user interaction. An approach of hierarchical 3D graph cuts is given, which initially extracts the object in the low reso-lution video, then projects the pulled objects to higher resolution video. The discussed method significantly reduces the number of graph nodes, and could achieve the high-quality extraction only with dependence on the user-supplied strokes on keyframe. The hierarchical 3D graph cuts could accelerate the efficiency of 3D Graph Cut. Frame-by-frame matting is difficult to achieve the spatiotemporally coherent video matting. An approach which formulating the video matting as solving the discrete poisson equations within space-time video cube. The frame trimap is produced by tracking the keyframe trimap. The biconjugate gradient is used for solving the banded sparse equations with millions of unknowns. The proposed method could produce the spatiotemporally consistent video matting.Extracting the natural shadow from an uncalibrated outdoor image is ill-posed, and it covers three subproblems: shadow determination, shadow extraction and shadow removal. A method of shadow extraction is put forward, which is based on the physical illumination model. Our approach approximately evaluates the illumination context, diffuse coefficient and illumination occlusion scalar. Compared with the previous methods, the given approach could extract the realistic shadow, remove the shadow simultaneously. Based on the characteristics of natural shadow, the shadow matte is smoothed, hence in the shadow removal image the texture attribute is well maintained and the shadow removal region is able to be seamlessly incorporated into the image.To compose the images taken from different illuminations leads to unrealistic composition. An image composition method with salience preserving is presented, which focuses on how to achieve the consistent luminance among composition components. The given approach includes salience determination, whitepoint correction and luminance adjustment with the constraint of salience. Firstly, the salience is evaluated on fully exploiting the difference of luminance and chrominance, then the principle color axes of the extracted image and background is aligned, finally the image composition is formulated as a nonlinear optimization problem with the salience constraint. The discussed schema could achieve the luminance consistent image composition with preserving the salience.