Research On Image Super-resolution Algorithm And Hardware Implementation

In this paper, the motive, significance and status of the research on image super-resolution processing are introduced at first, upon which, several single-frame and multi-frame image super-resolution algorithms, adapting for different applications, are proposed. Then fundamental theories in mathematics, verification methods and evaluation criterion of the algorithms are developed in turn. Finally, implementations of the algorithms in software and hardware are designed.Although the single-frame image super-resolution processing is a typical ill-posed problem in principle, it still meets the requirements of the applications, in which the source image has high signal-noise ratio and small scaling up ratio. Especially it is widely adopted in real-time image enlargement and the ASIC (Application Specific Integrated Circuit) design for image processing. In this thesis, some traditional linear interpolation algorithms and their frequency characteristics are reviewed, including ideal interpolation, nearest interpolation, bilinear interpolation, four-point and six-point bicubic interpolations. Much research has pointed out that the lower order interpolations achieve unpleasant visual effects, the higher order interpolations have higher complexity, and the algorithms based on statistics and set theories are unfit for implementation by hardware. To avoid the shortcomings of these methods, an adaptive interpolation algorithm based on Newton polynomial, with lower complexity and more pleasant visual effects than those of the bicubic interpolation, is proposed and verified.By multi-frame image super-resolution algorithms, the similar but incompletely uniform information contained in sequential images is used to restore a high resolution image. These algorithms are usually classified into the processing in frequency and spatial domain respectively. Since spatial domain restoration simultaneously involves interpolation, iteration, filtering and re-sampling etc., it can be described by more comprehensive imaging model. Therefore, a multi-frame image super-resolution algorithm based on Delaunay triangulation in spatial domain is proposed. Spatial irregular samplings points are triangulated by Delaunay method, then the gradient values on these vertexes are computed, and each triangular area is fitted into a continuous and continuously differentiable curved surface. High resolution image with discretional enlarged ratio is achieved by re-sampling the curved surface.To verifiy the validity of multi-frame image super-resolution algorithms, a camera motion model is designed. According to the model, a group of low resolution images are generated from ideal high resolution images. Thus the registration parameters of low resolution images are obtained on high resolution grid, neglecting other influencing factors on image registration. So the verification is more precise. The following contrast experiments have been conducted by several groups of low resolution images generated by the above method. Firstly, two groups of low resolution images are restored with the same frame number and different resolutions. Secondly, two other groups with the same resolution and different frame numbers are restored. The experimental results show that the quality of the source low resolution sequential images has decisive effect on restoration of high resolution images, and on the other hand, more source images are involved, more pleasant visual effect can be achieved in high resolution image restoration. However, exceeding certain frame number, the visual effect is improved slightly.Image quality assessment is an important means to evaluate the performance of image processing systems and algorithms. It involves image quality subjective assessment and objective assessment. The objective assessment is usually adopted in lab for its defective practicability. At present the classical objective assessment parameters, such as PSNR (Peak Signal-Noise Ratio) and MSE (Mean Square Error) based on error statistics, are widely adopted to assess the quality of degenerated images. In some cases, the evaluation results are coincident with results of the subjective assessment. However, objective assessment is ineffective in many applications. Therefore, a novel objective assessment method based on image structure information is proposed. Several images are assessed by PSNR and useless results are gained, but the assessment results by the method based on image structure information is in accordance with the subjective assessment as expected.The non-real-time processing of super-resolution images can not meet the requirements of actual applications, and the needs of ASICs for super-resolution image processing is increasing. Due to the limited research foundation, the hardware implementation of multi-frame image super-resolution has not been further researched, while for hardware implementation of single-frame image super-resolution, the following research has been done based on LCD Scaler. Firstly, by analyzing the research in quo and development trend of panel display technology, system architecture of LCD Scaler is designed. Then timing constrains of the Scaler are deduced, and the hardware structure of implementing the bilinear interpolation and the adaptive Newton interpolation algorithms is devised. The two algorithms are both implemented in Verilog HDL, and functional simulations are performed.In order to reduce the risk and cost of ASIC design, FPGA verification is usually adopted. The appropriate data as the system input are selected, and the corresponding scheme for data format conversion is proposed. Then the object code is downloaded into FPGA on the evaluation board. Finally, the whole system function is verified by FPGA.