Perception Guided Layered Depth Recovery From Single Images

Understanding 3D world in images is a basic problem in computer vision, which can be applied to many of industry and academic fields, including robotics, virtual reality, intelligence control and digital entertainment. Nowadays digital images are easy to capture and over billions of images uploaded to Internet every day. Thus, recovering 3D information from images, especially single images, becomes a hot research problem. However, it is an ill-posed problem to recover 3D information from single images because of inter-occlusion and intra-occlusion of objects. Fortunately, prior knowledges of perception and recognition give hints to recovering important 3D information from single scenes.This dissertation focuses on the fundamental problem, depth ordering, in recovering 3D information. Starting from human perception theory, we propose two depth ordering representations, and apply them in recovering 3D information, including bas-relief and 3D hallucinating stereoscopy. Furthermore, we apply depth ordering representations into a basic image understanding problem in computer vision-saliency retargeting.Main contributions of this dissertation are presented as follows:1. Perceptually oriented depth ordering representationsBased on two kinds of image descriptors, we present two novel image depth ordering representations, including contour based depth ordering representation and object based depth ordering representation. We analyze geometric elements (including regions, segments and junctions), and construct a two-level graph for depth layering according to relationships between geometric elements. We also employ an MRF formulation to encourage global consistent using perceptually depth cues (especially monocular depth cues) and adjacency between two objects; this representation has high accuracy in estimating both local and global depth orders.2. A single image based bas-relief generation algorithm.Based on contours, intensity and gradient information extracted from images, we propose a feedback method to iteratively reconstruct bas-relief, which can increase details on the bas-relief surface as well as preserving global depth layering information.3. A single image based stereoscopy hallucination effect algorithm.We propose an algorithm for efficiently hallucinating stereoscopy from single segmented images. We provide a novel method for depicting 2D shapes and automatically completing 3D objects based on symmetric and convex theories. Moreover, an iterative ground fitting method is provided for locating rough models in the scene. Not only users can experience stereoscopic hallucinations via auxiliary devices such as red-blue glasses and polarizing filter, but also via naked eyes directly from videos. Although we do not attempt to recover accurate 3D scenes, our results produce perceptually natural 3D hallucinations.4. Self-adaptive saliency retargeting strategyWe research into the relationship between depth ordering and image saliency retargeting, and propose a self-adaptive saliency retargeting method based on geometric locations and depth orders. In this algorithm, we analyse how depth orders and geometric locations play a role in object saliency, and formulate them for saliency editing. Our results show that the proposed method can retarget human attention to the modified image object, and the modification is hardly to be noticed by users.