Research On Super-resolution Reconstruction Of Multi-view Remote Sensing Images

Remote sensing images are always blured because of transmission, subsampling, relative movement et al.. Image deblur from a single image can improve the quality of remote sensing images. But the effect is not significant, since the information used for image enhancement is limited. With the development of remote sensing techonology, we can gain several images of the same scene in a short time. Then it is possible to estimate the latent sharp image of the scene by employing super-resolution methods. This is of important meaning to reveal the deep information of the images and improve the precision of remote sensing application. In the thesis the author use'multi-view'to distinguish between remote sensing images and multi-frame images of ordinary video. And multi-view remote sensing images include three-line camera images, images of contiguous tracks, images of aerial video cameras, images of satellite constellation et al..The imaging environment of remote sensing is more complex than ordinary video, and this causes some difficulties for super-resolution of multi-view remote sensing images. In particular, duo to a variety of factors leading to blur image, it is challenging to estimate the blur kernel. The complex deformation and gray difference among multi-view images increase the difficulty of image registration. To address these concerns, the main contents of the thesis are as follows.1) Kernel estimation is one of the key technologies of super-resolution. la related researches, blur kernel is modeled by a Gauss function with specified parameters. And the paremeters are empirical values, or set by experiments, or estimated by MTF. However, image blur is cuased by both internal factors of remote sensor and external factors of imaging environment, so it is limited to model blur kernel by a Gauss function. Thus an edge prediction based non-parametric method is proposed in this thesis to estimate blur kernel from a single remote sensing image. In the method shock filter is applied to sharp the current latent image, and then masks are used to select useful gradients for kernel estimation. As the estimated kernel is not sparse, an ordered region growing method is applied to extract the main structure of the kernel. Then the kernel is used for estimation of a deblurred image. During the iterations, more and more edges are included in the masks by decreasing the thresholds of masks, while the edges caused by ringing artifact are excluded. The performance of the proposed method is firstly verified by simulated experiments, followed by experiments with real remote sensing images of SPOT5, Tiangong-1 and ZY-3. The results show that the quality of the images is improved, and the edges are enhanced.2) One of the preconditions of super-resolution technology is that multi-frame images must contain different information about the same scene. Thus the motion estimation of multi-frame images must achieve subpixel accuracy. The deformation of multi-view remote sensing images caused by terrain fluctuation and the pose of sensor is non-global. And the common affine transform always fail to register multi-view images with high accuracy. An optical flow based method is proposed in the thesis for super-resolution reconstruction of multi-view remote sensing images. Optical flow is not restricted by parametric model, and can estimate arbitrary motion between images with subpixle accuracy. This meets the requirement of multi-view images registration. In the proposed method, optical flow fields of multi-view images are firstly used to estimate the original value of high resolution image, and then used for the reconstruction of high resolution image. During the iteration, optical flow fields and high resolution image are both updated. Chang'E-1 three-line images and infrared images of aerial video camera are used in the experiments. And the results of the proposed method are compared with affine transform based method. It is proved that edges are greatly enhanced in reconstructed images without appreciable ringing artifact. And some details which are unidentifiable in original images are recognizable in reconstructed images.3) Gray difference is ignorable for ordinary multi-frame super-resolution reconstruction. But gray difference of multi-view remote sensing images, which results from different imaging time, external environment, pose and position of sensors, can influence both the accuracy of image registration and the quality of reconstructed high resolution image. For this reason, a NCC-POC method which is robust to gray difference is proposed in the thesis. In the method, NCC is applied to adaptive window pair that goes through the image pair pixle by pixel to determine integer movement, followed by POC to estimated subpixel movement with high accuracy. Then the overall movement of the window pair is set to be the movement at the center pixel of the window. Finally a dense high accuracy movement map is generated, and post-processing is employed to eliminate invalid values. The movement map is used for both multi-view image registration and gray correction. And finally, a BTV-based super-resolution algorithm is used for debluring. In experiments, restoration of infrared images captured by aerial staggered TDI camera which suffer from gray difference and stagger displacement between the odd field and even field is treated as super-resolution reconstruction of the odd and even field image. Experimental results show that the quality of the images is improved obviously by both subjective and objective evaluation. The displacement is compensated effectively. Texture and edges are sharper and clearer.