Dichromatic Reflection Separation In Computer Vision

Changes in illumination can induce significant variations in the appearance of an object. With the research on illumination reflectance, people realize that reflection of light from surfaces can be classified into two broad categories: specular and diffuse. The specular component is highly dependent on the viewing direction, while the diffuse component is closely related to the material of the object but changes little when observing direction varies. In computer vision field, many researchers acquire useful information through the analysis of these two components, such as the information about the object shape and the illumination environment, etc. On the other hand, the mixture of specular and diffuse reflections can result in significant inaccuracy in a vast majority of techniques in areas such object recognition, segmentation, etc. Due to the reasons above, people began to pay efforts to separate these two components by making use of their inherent properties, which is so called dichromatic reflection separation problem.This thesis focuses on how to separate these two reflection components, based on the visual information in a single static image without any knowledge of objects' geometric parameters, surface material information or environmental lighting conditions. Because mathematically the separation process is ill-posed, it is impossible to accomplish this problem analytically in one action. Thus, in this thesis, the effort is focused on solving the dichromatic reflection separation problem on transparent surfaces (cornea of human eyes, glass, etc.). In this case, the transmitted light could be considered as diffuse, and reveals the environment details behind the surface, while the surface specular reflections shows the details of the frontal scene. Since the two components are usually very complex, previous separation methods are not applicable. So, in this thesis, three new methods are proposed: for corneal, a method uses physical properties of irises is proposed; for transparent material like glass, a local method based on junction features is proposed; due to the inefficiency of local features in global control, a semantic information based method is proposed. Experiment results show that these methods have good performance and could deal with more complex situations than the previous ones.In this thesis, the background and the motivation of the dichromatic reflection separation problem is explained firstly. And previous methods are discussed and compared briefly by pointing out their advantages and disadvantages. The challenge of this problem is also mentioned as an important part.Then, for the problem of transparent surface reflection separation, the thesis proposes three algorithms.For human corneal image, iris textures are considered as diffuse, and the environment illumination information contained in the specular component is significantly obscured by the iris textures. Because of the intricate textures and complicated illumination environment, previous methods could not applicable to corneal image dichromatic separation. So, in chapter 3, a method utilizes the physical characteristics of human irises (iris chromaticity, radial autocorrelation of iris texture, illumination correspondence between two irises) is proposed. Experiments show that after the separation, the specular component could give the illumination information more accurately.For ordinary transparent material image (e.g. glass), the corneal image separation method is not suitable any longer. The separation problem is more complicated. The two components reflect the scene information of the material surface's two sides, and form two transparent layers. In chapter 4, the dichromatic separation process is transformed into the gradient separation of two layers, and the local junction features are analyzed. A probability model is proposed to classify the junctions, and these junctions are first separated using linear transform, then, they are used as the other areas' constraints. After the gradient reconstruction step, the two separated gradient maps could be transformed back into the two layers.Because of the inefficiency of local features in global control, in chapter 5, a semantic information based method is proposed, in which the gradient description is still used. This method divides the image into patches, and by learning from the training data, SOLDA probability model is constructed to describe the relationship between different semantic categories and patches. When doing the separation, the semantic information of the two separated layers are used to globally constrain the patch distribution within the layers. Gibbs sampling is adopted to give the solutions when the layers' semantic information is known, or undetermined, in which case the layers' sementic categories are the bonus of the separation process.