Study On Methods Of Natural Color Night Vision Based On Image Analysis

With optical-electro imaging devices, the night vision technology expands the range of human spectral response and enhances the ability to observe in the dark, resulting in "transparent night". Until recently, the standard representation of night vision imagery is monochrome, which is disadvantageous to scene interpretation and target detection. However, with the deeper understanding of the color’s role in human visual perception, there is a growing interest in displaying night vision imagery with colors, especially with the natural colors that are consistent with human visual properties. The natural color night vision technology allows night vision images to obtain the best observing performances, thus has demonstrated important applications in both military and civilian areas, such as battlefield surveillance, intelligence transmission, criminal investigation, safety inspection, traffic control, night navigation, historic preservation.In this thesis, discussions on the natural color night vision technology are conducted in two aspects. The first aspect is focusing on the properties of night vision imaging and application, discussing about the problem of depth perception enhancement in night vision imagery, the problem of colorization for single band thermal images, and the problem of real-time colorization for visible/thermal images. The second aspect is focusing on the related mathematical models for the natural color night vision technology, discussing about the dimensionality reduction method for image recognition, and the sparse learning method for model training. Specifically, the main contributions and innovations of this thesis are as follows:(1) Study on Coloring night vision imagery for depth perception.Depth perception for night vision imagery is important for scene comprehension. A novel scheme is proposed to give multiband night vision imagery a natural color appearance with depth sense. The scheme simulates the depth cue by varying saturation value of each color object, in correspondence with its relative depth value that is estimated from the ratio between infrared and low-light-level sensor outputs. In the proposed scheme, a night vision pattern database is built in advance, and employed to recognize the night vision objects based on emitting-reflecting and texture features. Then, each object is provided its correspondence natural color, and the saturation value of the natural color is varied according to the estimated depth information. Experimental results show that the proposed scheme can achieve satisfying results, provides night vision imagery smoothly natural color appearance as well as the sense of depth, thereby the situational awareness and target detection can be improved.(2) Study on Colorizing Single Band Thermal Night vision Images.Consider the problem of assigning single-band thermal night vision image with natural day-time color appearance. Modeling color distribution of thermal imagery is a challenging problem, since there are insufficient local features for estimating the chromatic value at a point. The proposed color estimation model incorporates multi-scale and spatially arranged image features, both linear and nonlinear model (SVR) are discussed. A supervised learning procedure is first employed to estimate colors of monochromic images, since the irrelevant luminance between thermal image and natural image. Different from current color night vision methods that based on "multi-band night vision image fusion", the proposed approach can be directly applied on single-band thermal image, so that the portability of night vision system can be enhanced. Experimental results show that the proposed approach leads to relatively accurate description of the desired color distribution.(3) Study on Real-time Color Night-vision for Visible and Thermal images.A real-time scheme to display the visible and thermal images in a fused representation with natural daylight color background and highlight thermal targets is proposed. The scheme is based on a specially designed "natural- highlight color" lookup-table (LUT). The LUT is derived from the combination of a visible-thermal image and its daylight-background-highlight-targets fused representation. To form this representation, the grayscale visible image is first transferred daylight colors by using the natural color transfer technique based on local texture information, then the luminance component of this colorized visible image is replaced with the feature level fusion image that optimized the target region feature of thermal image and the texture feature of visible image by using K-means clustering algorithm and discrete wavelet transform (DWT). Once the LUT has been derived the color mapping can be applied to different images and deployed in real-time. Experimental results show that the proposed scheme can achieve satisfying results, the overall scene recognition and situational awareness can be improved.(4) Study on Global Inference Preserving Projection for Semi-supervised Discriminant Analysis.A new linear dimensionality reduction approach, Global Inference Preserving Projection (GIPP), is proposed to perform classification task in semi-supervised case. It is based on a new global structure, which reveals the underlying discriminative knowledge of unlabeled samples. A path-based dissimilarity measurement is used to infer the underlying class information for unlabeled samples. Experimental results on data visualization, object recognition database, face recognition database and spoken letter recognition database demonstrate the effectiveness of the proposed approach. Moreover, it is shown that the proposed approach can be successfully applied in colorizing low-light-level night vision image, in the sense that the natural color reference image for color transfer can be automatically and efficiently selected.(5) Study on A Variational Bayesian Approach to Sparse Linear Model Based on Double Lomax Priors.A new family of sparsity-promoting prior coined to as Double Lomax prior is proposed. It is shown that on one hand it provides a tighter approximation to the L0 norm than ARD prior thus has theoretical superior for recovering sparse vectors with fewer measurements; on the other hand it owns Gaussian Scale Mixture representation thus has computational tractability for efficient Bayesian processing. A full Variational Bayesian inference is developed here to solve for SLM using Double Lomax priors. Being a strictly log-convex prior, Double Lomax prior brings challenges in inference procedure, such as multi-mode and asymmetrical posteriors. Analysis shows that the Variational Bayesian inference developed here is needed for avoiding local minimum and over-fitting. Experiments on both correlated and uncorrelated SLM simulations with applications to AR model identification and compressive sensing have demonstrated the effectiveness of the proposed approach. Moreover, it is shown that the proposed approach can be successfully applied in colorizing single band thermal night vision images, in the sense that fewer training samples are needed.