Video Restoration Based On Low-rank Matrix Recovery

Video restoration is a hot issue in digital image processing area. In the past, all of the image and video data are captured by film-based camera. Due to the very long time and improper storage, chemical change will occurred in many old films, which results in various kinds of distortion in videos like spots, noise, scratches and faded etc. Now typically we can use analog-to-digital conversion in order to save some precious historical materials. However, it’s hard to avoid adding electronic noises during the process. This leads to a further reducing of video quality. Therefore, for the rescue of the historical data or improving the quality of images in daily life, digital image and video restoration is necessary and important.In recent years, low-rank matrix recovery has becoming a hot research area of signal processing, computer vision and artificial intelligence fields. The theory of low-rank matrix recovery derived from compressed sensing, which is a very popular signal processing method. However, after the study of many scholars, it has gradually formed its own set of theoretical system and becoming a very effective tool for signal processing, which achieved very good performance in many applications. This paper investigates the theory, algorithm and applications of low-rank matrix recovery and focus on how to use this effective tool to solve video restoration problem. Then we proposed a new video restoration approach. By using a modified version of random PatchMatch algorithm, nearest-neighbor patches among the video frames can be grouped quickly and accurately. Then the video restoration problem can be boiled down to a low-rank matrix recovery problem, which is able to separate sparse errors from matrices that possess potential low-rank structures. Furthermore, the reweighted low-rank matrix model is used to improve the performance of video restoration by enhancing the sparsity of the sparse matrix and the low-rank property of the low-rank matrix. Experimental results show that our system achieves good performance in denosing of joint multi-frame and inpainting in the presence of small damaged areas. The main contribution of this paper is shown as follows:(1) A new video restoration approach was proposed in this paper. By using a modified version of random PatchMatch algorithm, we boiled the video restoration problem down to a low-rank matrix recovery problem. Our video restoration system achieved good performance in denosing of joint multi-frame and inpainting in the presence of small damaged areas, compared to many state-of-art methods. (2) We modified the version of random PatchMatch algorithm from2D to3D, which can group nearest-neighbor patches among the video frames quickly and accurately. The modified version of random PatchMatch algorithm is very suitable for video processing.(3) The reweighted low-rank matrix model is used to improve the performance of video restoration by enhancing the sparsity of the sparse matrix and the low-rank property of the low-rank matrix. A corresponding reweighted Inexact Augmented Lagrangian Multiplier (IALM) method was designed to solve this reweighted model and it achieved better performance than many present algorithms.