Study On The Reconstruction Of Spatial And Temporal Super Resolution Image Based On Bandelet Transform

Visual perception of image and video depends on the temporal and spatial resolution of image. The higher the temporal resolution is, the more natural will be the moving image in video looks. The higher the spatial resolution gets, the more obvious will details appear. In order to meet people's sensory enjoyment or need for image information, super resolution technology is designed to improve temporal / spatial resolution of image and video by software.At present, there are lots of domestic and oversea studies on super resolution technology. These studies may be divided into two categories: temporal super resolution technology and spatial super resolution technology. Temporal super resolution technology is the way to insert a frame of image(middle frame) from its front and rear frame so as to improve the temporal resolution of vedio sequence; while spatial super resolution technology is to produce a higher spatial resolution image from one or multiple low resolution images. Many research achievements have been obtained both in temporal and spatial super resolution technologies. However, they are still facing a number of key problems unresolved. For example, in temporal super resolution technology, precision and accuracy of motion estimation are not perfect and the corresponding middle frame image is not clear in some local area. While in spatial super-resolution technology, with the increasing requirement on image resolution, edges of image may easily get blurred and toothed. On the other hand, Bandelet transform is a wavelet transform beyond, which may adaptively construct Bandelet bases according to textures of an image, so the best sparse reprent of the image may be obtained. Therefore, Bandelet transform was applied to the field of image compression by researchers, and good results have been obtained. In contrast with traditional way, it was found by the study in this thesis, that the adaptive edge tracking features of Bandelet may also be applied to the field of image super resolution.Based on the study of super resolution technology, the edge tracing feature of Bandelet transform was utilized to overcome the problems in conventional super-resolution technology, and the temporal / spatial resolution of image and video were improved in this thesis. The main contents include the following aspects:1. In order to improve the accuracy of motion estimation and overcome the shortage of traditional block-matching criterion, a method of motion estimation based on prior confidence level and matching confidence level was proposed. With this method, the speed and accuracy of matching may be increased, and the possibility of false matches may be reduced. Firstly, the stationary part and moving part of an image is separated by the detection of still image, and then by methods of target blocks sorting through prior confidence level, motion vector recursion, locally full search of matching confidence level, a rapid motion vectors estimation is obtained. The motion vector filtering and overlapped block based motion compensation were also used to achieve the temporal super resolution image reconstration.2. Conventional motion estimation criteria are based on pixel block consistency judgment without taking into account the texture and visual characteristics of image. The advantages and disadvantages of image similarity criterion was firstly analyzed in this thesis. Since the direction of texture edge after Bandelet transform may be accurately obtained, a geometry similarity criterion, which is possible to make block matching from the image geometry(side length, texture) properties was proposed. A similar block matching criterion based on human visual effects was proposed by making use of similarities between Bandelet multi-scale decomposition, direction adaptive characteristics and features of human visual system(linear or quadrature phase, shift invariance, multi-scale, sensitivity to directional stimulation).3. In current super resolution image reconstruction algorithms for a single image, such as edge protection interpolation algorithm and wavelet edge interpolation algorithm, due to the limitation on direction, they cannot take full advantage of the unique geometric features of the image itself. A single image super resolution reconstruction algorithm based on Bandelet transform was proposed in this thesis. The edge adaptive feature of Bandelet transform is utilized in image partition. So as different regions are interpolated with different interpolation algorithms, particularly in edge regions, coefficients projection and rearrangement along the geometric edge direction are adopted, and the coefficient rearranging convergence in the texture direction is utilized to achieve spatial super resolution reconstruction of single image.4. Because both noise and image edge contain high frequency components, it is a paradox to keep image edge character and reduce noise influence in image super-resolution reconstruction process. In order to solve this problem, considering the advantages of both Bandelet transform and Steins unbiased estimation(SURE) algorithms, a new threshold function which combines the features of soft threshold and hard threshold function, was proposed. Then, a sparse expression of image was obtained through the Bandelet transform. The noise was reduced by minimizing the mean square error between the noised image and the predicted one.