Research On Super-Resolution Reconstruction

The spatial resolution that represents the number of pixels per unit area in an image is the principal factor in determining the quality of an image. With the development of image processing applications, there is a big demand for high-resolution (HR) images since HR images not only give the viewer a pleasing picture but also offer additional detail that is important for the analysis in many applications. The current technology to obtain HR images mainly depends on sensor manufacturing technology that attempts to increase the number of pixels per unit area by reducing the pixel size. However, the cost for high-precision optics and sensors may be inappropriate for general purpose commercial applications, and there is a limitation to pixel size reduction due to shot noise encountered in the sensor itself. Therefore, a resolution enhancement approach using signal processing techniques has been a great concern in many areas, and it is called super-resolution (SR) (or HR) image reconstruction or simply resolution enhancement in the literature. The basic idea behind SR is the fusion of a sequence of low-resolution noisy blurred images to produce a higher resolution image or sequence. Early works on SR showed that the aliasing effects in the high-resolution fused image can be reduced or even completely removed, if a relative sub-pixel motion exists between the undersampled input images. However, contrary to the simple frequency domain description of the early work, in general, SR is a computationally complex and numerically ill-posed problem. This makes SR one of the most appealing research areas for image processing researchers. For some problems existing in the field of SR reconstruction, such as image registration, motion estimation and regularization constraints, we investigate the SR problem from the following aspects, and propose some novel and improved ideas and methods with respect to SR image reconstruction, and thus develop SR technology further. 1. This paper presents a novel method of image registration based on analytical Fourier-Mellin transform (AFMT) for computing transformation parameters from two images that differ by a geometric transformation. The geometric transformation consists of x-axis and y-axis translation, rotation, and uniform scale transformation. The basic idea of the proposed image registration method is that the transformed image after an appropriate transform should be variant to one transformation parameter, but invariant to the other parameters. Thus, it is very suitable to use the transformed image to computer each of geometric transformation parameters individually in succession. The properties of Fourier transform indicate that Fourier magnitude spectrum is invariant to translation, but variant to rotation and uniform scale. On the other hand, AFMT is a transform oriented to rotation and uniform scale, while modulus based AFMT is variant to scale. Therefore, these properties lend themselves to the determination of scale factor by two serial transforms, Fourier transform and AFMT. Obviously, the image registration method based on AFMT can be used to find a set of appropriate transformation parameters. Since the original method is based on the discrete log-polar coordinates, the numerical conversion from Cartesian to log-polar coordinates of the discrete image is required, and will bring significant resampling error, which severely interfere the resulting transformation parameters. Furthermore, the error introduced by the interpolation may reduce the accuracy of registration. As a result, it is necessary to control the diffusion of error. For the purpose, the proposed registration method aiming at finite discrete images adopts the Cartesian AFMT approximation. The Cartesian approximation can eliminate the conversion between the two coordinate spaces, so the proposed method can avoid the process of interpolation required in the conversion and the influence of the resampling error, reduce the registration error and obtain a significant improvement in the aspect of accuracy and robustness compared with the conventional method using cross-correlation functions. In the implementation of registration method, in order to remove "+"shaped artifact in the spectrum, we propose a border blurred filter, which is applied to our proposed algorithm. By the operations in spatial domain, the border blurred filter can improve the frequency components and eliminate "+"shaped artifact in frequency domain. With this filter, very few pixels need be altered. In general, within a few pixels of each border there is some effect, so that a majority of pixels are unchanged. In experiments, the two results of normalized high frequency energy and mean squareerror both show that the pre-filtering with the border blurred filter makes significant improvement on the reduction of "+"shaped artifact, without changing the most of the original image information and the most of Fourier spectrum coefficients. 2. We propose a novel optical flow based motion estimation technology, which is combined with Bayesian multiframe SR reconstruction algorithm to achieve Bayesian SR reconstruction. Optical flow based motion estimation adopted in the paper is a hybrid method with a class of fourth-order partial differential equation constraints. This method embeds Lucas-Kanade local technique into Horn-Schunck global approach, yielding a new technique with dense optical flow fields, which is robust against noise. The use of the new technique doesn't require post-processing for the interpolation of sparse data. Therefore, this method has the features of local spatial constancy of optical flow fields for Lucas-Kanade method, and the features of spatiotemporal constancy for Horn-Schunck method due to the combination of different smoothness effects from local and global methods. In the proposed method, we also exploit a fourth-order partial differential equation constraint term to improve smoothness constraints, and to avoid over-smoothness effect in optical flow fields. Moreover, the use of nonlinear penalty functions or nonlinear penalisers can give better flow discontinuity-preserving. Both of them contribute to the precision of motion estimation, and the improvement of SR reconstructed image quality. In order to implement the computation of linear and nonlinear approximation equations of the hybrid method, and approximation equations with fourth-order partial differential equation constraints, we adopt an iterative method called successive overrelaxation. The iterative method uses less storage space with higher convergence rate. The experimental results show that the precision of motion estimation has direct and significant effect on SR reconstructed image quality. In addition, we present a generalized mathematical framework for optical flow computing. In the framework, the adoption of spline functional as smoothness constraints contains the two special cases of second-order and fourth-order partial differential equation constraints. Moreover, we prove the necessary condition of the solution of the minimization problem for optical flow computing. Actually, the process of the proof is a process to solve the minimization problem for optical flow computing. In methodological point of view, the proof provides the foundation for the formulation and numerical computation of optical flow computing. Another task of the mathematical framework is the reduction from the partial differential equations to linear algebraic equations with respect to two motion components u and v. Theconversion contributes to the discrete formulation of optical flow equations, and at the same time, in algebraic point of view, proves that the use of smoothness constraints guarantees the existence and the uniqueness of the solution 3. In the most of medical image sequences, the image contents change with time elapsing, consequently, the image intensities vary among frames. The direct SR reconstruction from each single frame will result in the magnification of noise, or the loss of spatial detail information in the case of pre-filtering for smoothing, while the traditional SR reconstruction from multi-frame will lead to intensity degradation due to the inconsistency of image intensity. For the solution of the problem, we propose a SR reconstruction scheme with the feature of intensity preserving. In SR reconstruction, we employ a novel intensity-preserving filter, which is constructed based on a bilateral filter. Different from the traditional filter such as Gaussian filter, the bilateral filter defines the closeness of two pixels not only based on geometric distance but also based on photometric distance. Therefore, the filtering result isn't only influenced by geometric distance, but also by photometric distance. We extend the traditional bilateral filter to achieve the generality by replacing the central pixel with the pixel of interest (POI), and providing other possible convolution kernel for the filtering of support domain and intensity range. The improved bilateral filter has the following features: 1) a simpler kernel than the Gaussian adopted in the bilateral filter may be easier to compute, leading to a performance speed enhancement; 2) selecting the POI rather than the center pixel leads to a general filter; 3) the improved bilateral filter enhances the ability to control the weights of pixels within the support; 4) by defining an appropriate POI, the improved bilateral filter extends the ability of controlling outputs, and broadens the extent of its applications. From the extension, we note that the improved bilateral filter can be spatial distance weighted or photometric distance weighted filter (controlled by the parameters σd and σr), linear or nonlinear filter (determined by the choice of POI), spatial or temporal filter (determined by the size in each dimension of 3D support ?). In order to achieve the two goals, suppressing noise as well as preventing the high intensities from being degraded, we define the POI using maximum aggregate functions. Through the definition, more high-intensity values can be preserved after filtering. Therefore, in addition to denoising, the intensity-preserving filter can also avoid the degradation of the image intensity. In this paper, we propose a SR reconstruction scheme for fluorescein angiogram sequence, which combines the new intensity constraint to Miller's regularizationformulation to achieve the ability of preserving high-intensity. In reconstruction, intensity-preserving filter plays a role of pre-filtering noise and determining intensity template. In implementation, we use a modified Q-th order converging algorithm to solve the minimization problem subject to both the smoothness and the intensity constraint. The intensity template derived from a pre-filtering process is employed in the intensity constraint, representing an image with the desirable intensities. The estimated SR reconstruction may be influenced by the regularization parameter of the intensity constraint as does the regularization parameter of the smoothness constraint. From two aspects, we explain the importance of the proposed intensity-preserving SR reconstruction scheme. On the one hand, the proposed intensity constraint is a novel regularized constraint used to preserve intensity, which enriches the contents of regularized constraints; on the other hand, the scheme is an application of SR reconstruction in the medical imaging field of fluorescein angiogram sequences, which broadens the extent of the SR applications. Moreover, the idea of intensity preserving can further be applied in SR with significant illumination change. In conclusion, the achievement of the investigation results enriches the approaches to the SR problem, and has a certain theoretical and practical importance. This paper provides useful methods and approaches for the research and the development of SR reconstruction problem.