Tone-aware Image And Video Understanding And Editing

Due to the information richness and intuitive understandability,image and video have lots of applications in digital media,intelligent systems,social entertainment,webcasting,surveillance security and military investigation and other relevant fields.These aforementioned applications basically rely on accurate understanding and reliable editing to image and video.As the human visual perception system is highly sensitive to the tone,tone-aware image and video understanding and editing have attracted a lot of attention,and have been developed into a hot research topic in both computer graphics,computer vision and image and video processing.The main purpose of the topic is to help the computer or user fulfill tasks such as analysis,identification,cognition,adjustment and editing of image and video,and to generate the results that meet requirements of specific application through construction of suitable models and theories based on tone.In this paper,the theory,method,key technology and application of this topic are deeply studied.Specifically,the research is carried out in three aspects:cloud detection of aerial image,underexposed video enhancement and image recoloring.In particular,we first study the tone-driven content understanding problem,and explore the cloud detection of color aerial photographs.Next,we focus on the underexposed video enhancement and image recoloring with tone editing as the main body.Fast and reliable understanding and editing of image and video is a research topic that involves many difficulties and challenges.The procedure requires accurate dif-ferentiating of objects in the scene and inferring their categories and location,which essentially involves the scene semantic analysis,object recognition,image segmenta-tion and many other problems.Although researchers have made great progress in the field of these problems,efficient understanding of specific object is still a problem to be solved.There are many image and video editing approaches,involving color,illumina-tion,contrast,style and spatial-temporal location,etc.However,they commonly have low efficiency and poor usability,and fail to generate visually satisfactory results.Due to the high sensitivity of the human visual perception to tone,we in this paper use tone as clue,comprehensively study how to understand and edit image and video in three aspects,namely cloud detection of aerial photographs,underexposed video enhancement and image recoloring.Based on the tone difference between cloud and non-cloud region,we first study cloud detection of high resolution RGB color aerial photographs.Then,we analyze and discuss tone recovery in low light level environment,and explore the underexposed video enhancement problem.Finally,we study how to achieve intuitive,efficient,and realistic image recoloring by editing a simple color palette derived from the original image.Specifically,this paper has the following three main contributions:(1)we propose a cloud detection method for RGB color aerial photographs.Based on statistical characteristics of hue and texture of cloud region in aerial photographs,a coarse-to-fine cloud detection framework is constructed,which achieve high de-tection accuracy by effectively separating cloud and non-cloud ground regions.In addition,we introduce a local linear model for the first time to detect semitrans-parent cloud regions in aerial photographs.(2)We develop an underexposed video enhancement method based on perceptual-driven fusion,which achieves the purpose of virtual illumination compensation for underexposed video,and accordingly enhance scene visibility,contrast and color saturation.The proposed method makes the first attempt to incorporate the hu-man visual perception measures for assessing video quality,which helps guide the algorithm to generate the global well-exposed video.Besides,a perception-driven progressive fusion framework is proposed for seamless integration of all locally best exposed regions.Finally,in order to remove the noise interference and to avoid degrading texture structure,we further propose texture-preserving adaptive spatio-temporal filtering.(3)We propose a palette-driven.image recoloring algorithm based on color decompo-sition optimization,which greatly simplifies the image recoloring operation while ensuring that the generated results are consistent with human visual perception,with vivid colors and natural appearance.To efficiently summarize the main color categories of the original image into a compact color palette,we first proposed an effective palette extraction algorithm,which is independent of the image size.Then,we construct the color decomposition optimization framework for revealing thees-sential relationships between color palette and pixel colors.Finally,we introduce how to evaluate image recoloring algorithms,and firstly publish a image collection for evaluation and comparison.Focusing on understanding and editing image and video,we study several tone-related problems from different perspectives,and provides detailed solutions to key technical difficulties involved in each problem.The experimental results show that the proposed methods in this paper are superior to the existing approaches in their respective fields,and can be extended to a variety of image and video related application scenarios.