Research On Image And Video Super-Resolution Reconstruction

A high quality image always contains further detailed information of targets, and it is of great value for analysis and post-process. Along with the rapid improvements of video and image processing technologies in recent years, the demand for high resolution video and image sequences grows fast, which gradually raises new challenges for manufacturing technology on image sensors. At present, however, it is impossible or hard to break the bottleneck by hardware schemes, such as reducing the pixel size or increasing the chip size. In the meanwhile, prices of expensive high precision sensor are not applicable to wider applications. So, how to enhance the spatial resolution of video and image sequences under these limitations becomes an active research topic. The technique of super-resolution (SR) reconstruction is developed under this circumstance. Super-resolution reconstruction refers to a software technique that enhances the resolution of images or videos by using digital signal processing technology. It reconstructs high resolution and high quality image(s) from a group of degraded low resolution images with sub-pixel shifts from the same scene. It breaks though the resolution limit of image acquisition equipment and can achieve data fusion on sub-pixel level by wealthy complementary information. Utilizing SR techniques can improve the resolution of images without updating the existing low-resolution sampling devices; and also can enjoy high-quality videos without increasing the signal transmission bandwidth. SR reconstruction processing has proved to be useful in many practical applications, such as remote sensing, military detection, medical image, etc. Currently, image super-resolution reconstruction has become one of the hottest areas of image processing. With the deepening of the research, the range of this technique has expanded from traditional original image sequence to compressed video, from spatial resolution to temporal resolution, from mono-view video to multi-view video.The dissertation investigates several key issues of super-resolution of video and image sequences including image registration, stereo matching and spatio-temporal super-resolution for stereo video. The main contributions and innovation points of the dissertation are as follows:1. Super-resolution reconstruction usually consists of three steps:registration, reconstruction and restoration. As the necessary step of super-resolution reconstruction, the performance of the registration algorithm greatly affects the quality of the reconstructed images. This paper proposed a sub-pixel registration method using a combination of wavelet-based multi-resolution decomposition, NEDI interpolation and three-step search block-matching algorithm. The proposed approach can greatly reduce the search volume and accelerate the speed of the registration, while maintaining registration accuracy.2. Mutual information is a quantitative criteria based on the traditional Shannon entropy. It takes into account only the relationships between corresponding individual pixels and not those of each pixel’s neighborhood, such that the alignment of the peak is not sharp, the best alignment position is difficult to locate. We propose a new similarity metric, which combines mutual information and a weighting function based on image gradient and image variance. This method effectively joints qualitative spatial positional relationship and quantitative statistical information of grayscale. The proposed similarity measurement alleviates the local minima problem and is more robust to noise than only with mutual information.3. We propose a new local stereo matching approach of combined adaptive weight and mutual information, which incorporating the advantage of the former in cost aggregation, as well as the advantage of the latter in cost computation. The experimental results show that the accuracy obtained by our proposed stereo matching algorithm is almost as excellent as state of the art local algorithm. Especially, this method can effectively improve the performance with kinds of radiometric differences, which is difficult to solve by other existing algorithms.4. We propose a spatio-temporal super-resolution reconstruction method for a distributed multi-view video coding architecture. The super-resolution scheme achieves temporal resolution enhancement and spatial resolution enhancement for mixed resolution video by exploiting inter-view/temporal correlation. Simulation results indicate that the super-resolution method has achieved good results in subjective visual and objective performance.