Research On Enhancement Techniques For Image Sequences Based On Low Rank And Sparsity

As an entity that records, transmits, and stores information, digital image sequences (including images, and image sequences) play a crucial role in human perception, and are widely applied in the fileds of national defense security, geological exploration, agriculture, forestry, fishery, scientific research, and people’s daily life. However, image sequences are often influenced by some degradation factors during the procedures of acquisition, transmission, and storage, leading to quality deterioration. Thus, the enhancement techniques for image sequences are always a popular research topic in signal processing field, aiming to restore or enhance the quality of image sequences, from which people can obtain information more accurately and effectively.With the ongoing indepth research on enhancement techniques for image sequences, it is realized that fully exploiting the characteristics of image sequences is of great importance to improving the performance of enhancement techniques. Some significant characteristics of image sequences, such as low rank and sparsity, can be considered as convex constraint descriptions and applied to the formulation of image sequence enhancement problems. In the meantime, the booming development of convex optimization theory and algorithms also provides a strong support for studying these problems. In this dissertation, denoising and super-resolution are selected as two typical topics on image sequence enhancement and studied, based on exploiting the low-rank and sparse properties of image sequences. The main work and innovations of this dissertation are listed as follows:(1) A two-step denoising method for image sequences based on low rank and sparsity is proposed, which is of common use. Firstly, by analyzing the characteristics of image sequences, we find that the signal component shows very strong inter-frame correlations, while the noise component shows sparsity. After reorganizing a three-dimensional image sequence into the form of a two-dimensional matrix, the corresponding signal matrix has low rank, and the noise matrix has sparsity. Thus, in the first step of the proposed denoising method, signal and noise components are preliminarily decomposed, by reasonably constraining them with low rank and sparsity, respectively. Then, we consider the local transient signals that exist in image sequences. They violate the low-rank property of matrix, but with sparsity instead, so they are mistakenly decomposed into the noise component. In the second step of the proposed denoising method, we reorganize the prelimilary noise matrix back to a three-dimensional image sequence, in order to exploit the usual spatial correlations within images. Then each image of the sequence is further decomposed into a compensatory signal component and its noise counterpart, by respectively using spatial smoothness and sparsity as proper constraints for them. The final denoised image sequence is the superposition of the signal component in the first step and the compensatory signal component in the second step.(2) Two denoising methods for hyperspectral images and for biological cell calcium fluorescence image sequences, respectively, are proposed, both based on low rank and sparsity. On the basis of (1), hyperspectral images and biological cell calcium fluorescence image sequences are taken as two examples, to present the proposed two-step denoising method for image sequences, with respective specific refinements:on one hand, by carefully analyzing the characteristics of hyperspectral images, we find that the noise level fluctuates dramatically in different wave bands; that is to say, the sparsity of the noise component matrix is structured column-sparsity, leading to a constraint with mixed norm l2,1. The proposed denoising method for hyperspectral images exploits the low-rank property of signal and sparse property of noise. Experimental results have validated the high denoising performance achieved by the proposed denoising method for hyperspectral images. Moreover, the proposed method shows high universality, because it applies to different types of noise that exist in hyperspectral images, and it does not rely on any idealized assumptions. On the other hand, a large amount of photon noise randomly locates in the calcium fluorescence image sequences. Thus the noise component matrix has global uniform sparsity, which is enforced by l1norm. Further analysis indicates that the location and shape of cell structures remains steady, which inspires us to introduce a weight matrix W into the proximal point algorithm, in order to help distinguish photon noise from the flashing signal in the similar form of dot. Experimental results have shown that by exploiting the low-rank property of signal and sparse property of noise, the proposed denoising method for biological cell calcium fluorescence image sequences can effectively remove photon noise, and improve the observation accuracy of calcium signal.(3) A super-resolution reconstruction method for single image based on low rank and sparsity is proposed. Firstly, natural images often contain repetitive or similar structures. This nonlocal self-similarity of images is important prior knowledge. The matrix formed by a group of similar image patches has intrinsic low-rank property. By combining the low-rank property with image observation model, an initial reconstructed high-resolution image is obtained. Then, a vectorized image patch can be sparsely represented as a linear combination of a few atoms chosen from an over-complete dictionary. By means of enforcing the high-resolution and low-resolution image patch pairs to share the same sparse representations, a pair of over-complete dictionaries are jointly learned. We sweep all the patches of the image that is obtained in the first step, and recover their higher-resolution counterparts using the dictionary pair, then the reconstructed high-resolution image is refined in the second step. Finally, to prevent the reconstructed image from drifting off the observation model, a post-processing step is performed, and we restore the final high-resolution image. Experimental results have indicated that the proposed image super-resolution reconstruction method, based on low rank and sparsity, can guarantee the fidelity to low-resolution observation, repress visual artifacts (such as ringing and jaggies), meantime preserve salient edges and fine structures in the restored high-resolution image.