Font Size: a A A

Multi-Modal Saliency Fusion for Illustrative Image Enhancement

Posted on:2015-12-07Degree:Ph.DType:Dissertation
University:University of Maryland, Baltimore CountyCandidate:Morris, Christopher JoelFull Text:PDF
GTID:1478390017489625Subject:Computer Science
Abstract/Summary:
Digitally manipulated or augmented images are increasingly prevalent. Multi-sensor systems produce augmented images that integrate data into a single context. Mixed-reality images are generated from the insertion of computer generated objects into a natural scene. Digital processing for application-specific tasks (e.g., compression, network transmission) can create images distorted with processing artifacts. Augmentation of digital images can lead to the inclusion of artifacts that influence the perception of the image.;In an augmented image, visual cues (e.g., depth or size cues) may be perceptually inconsistent. A feature deemed important in its local context may not be as important in the broader integrated context. Inserted synthetic objects may not possess the appropriate visual cues for proper perception of the overall scene. In compressed images, finer cues that distinguish critical features may be lost. Enhancing augmented images to add or restore visual cues can improve the image's perceptibility. This dissertation presents a framework for illustrating images to enhance critical features. The enhancements, inspired by an analysis of artists' techniques, bolster the features' perceptual cues and improve the comprehension of the augmented image. The framework uses a linear combination of image (2D), surface topology (3D), and task based saliency measures to identify the critical features in the image. Upon identification, the features are interactively enhanced using a non-photorealistic rendering (NPR) deferred illustration technique. The use of multi-modal saliency allows a visualization designer to adjust the definition of critical features.;The proposed framework provides a generalized, flexible, and extensible approach to enhancing salient features in an augmented image. The framework describes a metric, the Saliency Similarity Metric (SSM), for providing feedback on how closely the salient features of the enhanced image match those of the reference image. This feedback can be used for making informed decisions on tuning the visualization. The benefits of the framework are analyzed through objective and subjective evaluations. The evaluations reveal that illustrative enhancements must be carefully applied for perceptual improvement. The framework provides the flexibility necessary to effectively tune the enhancements to a particular task, data set, or user.
Keywords/Search Tags:Image, Saliency, Framework, Critical features
Related items