Image Processing Based On Deep Learning

Due to the popularity of smartphones and micro-cameras,photography has become an indispensable part of daily life.The technique of repairing and enhancing image quality is called image processing.Image processing involves a range of techniques in computer vision and graphics.It is of great significance and value to improve the capability of photo capturing,post-processing and some related visual applications.In recent years,the rapid rise of deep learning has significantly promoted the development of image processing.This dissertation focuses on the classic low-level and mid-level image processing problems,and proposes several deep learning based techniques to solve these problems better.This dissertation specifically studies the problems about image smoothing,reflection removal,intrinsic image decomposition and parameterized image processing.It presents the weak supervised learning,unsupervised learning and decouple learning techniques for the first time to solve these problems.The contributions of this thesis are mainly as follows.1.Image smoothing via unsupervised learningImage smoothing represents a fundamental component of many disparate computer vision and graphics applications.In this thesis,we present a unified unsupervised(label-free)learning framework that facilitates generating flexible and high-quality smoothing effects by directly learning from unlabeled data using deep convolutional neural networks(CNNs).The heart of the design is the training signal as a novel energy function that consists of an edge-preserving regularizer which helps maintain important yet potentially vulnerable image structures,and a spatially-adaptive Lp flattening criterion which imposes different forms of regularization onto different image regions for better smoothing quality.We implement a diverse set of image smoothing solutions emploving the unified framework targeting various applications such as.image abstraction,pencil sketching,detail enhancement,texture removal and content-aware image manipulation.2.An edge-oriented single image reflection removal approachThis thesis proposes a deep neural network structure that exploits edge information in addressing representative low-level vision tasks such as layer separation and image filtering.Unlike most other deep learning strategies applied in this context,our approach tackles these challenging problems by estimating edges and reconstructing images using only cascaded convolutional layers arranged such that no handcrafted or application-specific image-processing components are required.Using a mild reflection smoothness assumption and a novel synthetic data generation method that acts as a type of weak supervision,our network is able to solve much more difficult reflection cases that cannot be handled by previous methods.3.A general and robust intrinsic image decomposition approachWhile invaluable for many computer vision applications,decomposing a natural image into intrinsic reflectance and shading layers represents a challenging.underdetermined inverse problem.In contrast to many previous learning-based approaches,which are often tailored to the structure of a particular dataset(and may not work well on others),we adopt core network structures that universally reflect loose prior knowledge regarding the intrinsic image formation process and can be largely shared across datasets.We then apply flexibly supervised loss layers that are customized for each source of ground truth labels.The resulting deep architecture achieves state-of-the-art results on all of the major intrinsic image benchmarks.4.Decouple learning for parameterized image operatorsMany different deep networks have been used to approximate,accelerate or improve traditional image operators.Among these traditional operators,many contain parameters which need to be tweaked to obtain the satisfactory results,which we refer to as"parameterized image operators".However,most existing deep networks trained for these operators are only designed for one specific parameter configuration,which does not meet the needs of real scenarios that usually require flexible parameters settings.To overcome this limitation,we propose a new decouple learning algorithm to learn from the operator parameters to dynamically adjust the weights of a deep network for image operators.Experiments demonstrate that the proposed framework can be successfully applied to many traditional parameterized image operators.