Digital Image and Video Denoising in Multimedia Applications

Noise reduction (denoising) is of crucial importance in multimedia applications, as digital images and videos are often contaminated by noise during acquisition, storage, and transmission, and accordingly denoising is required for visual improvement or as a preprocessing step for subsequent processing tasks, such as compression and analysis. The purpose of this research is to investigate novel algorithms for digital image and video denoising.;First we address the problem of image denoising. Despite the richness of the image denoising research, most existing work only handles the grayscale image, whereas color image denoising has received much less attention. Considering the extremely popular color image production and application nowadays, we invent a novel denoising scheme specially designed for color images called multichannel non-local means fusion (MNLF), building on the grayscale denoiser non-local means (NLM). We formulate the color noise reduction as a minimization problem and derives the optimal solution consisting of constructing multiple NLM estimates spanning all three channels and fusing them together optimally. Besides, the denoising problem is investigated jointly with the demosaicking problem for noisy color filter array (CFA) images. The NLM filter is improved and extended for CFA image denoising by taking into account the special layout of CFA images, and then the full color image is reconstructed by applying any demosaicking algorithm to the denoised CFA image.;Next, denoising problems of video signal are addressed. To fully take advantage of both the backward and the forward inter-frame correlation, we come up with a generalized multihypothesis motion compensated filter (GMHMCF), where the reference frames can include both the denoised previous frames and the noisy future frames. The GMHMCF achieves noise reduction through the linear mean squared error (LMMSE) filtering and the denoising performance of LMMSE using different configurations of reference frames are discussed and analyzed. Further, we address the extension of GMHMCF to color video denoising by two approaches. The first approach decorrelates the RGB signal by converting the RGB samples to a linear luminance-chrominance space, and derives an adaptive optimal space to achieve the minimum overall denoising error. We call this approach as LAYUV. The second approach named CIFIC (standing for combined inter-frame and inter-color prediction) takes advantage of the inter-color correlation directly in the RGB space by introducing the inter-color prediction, that is, forming multiple predictors for each color component using pixel intensities of the current frame and motion-compensated neighboring frames both in this color component and the other two color components. We revisit and reformulate LAYUV and deduce that LAYUV is a restricted version of CIFIC, and thus CIFIC can theoretically achieve lower denoising error than LAYUV.;Finally, we address a special kind of noise, film grain noise, which is caused by the developing process of silver-halide crystals dispersed in photographic emulsion. Film grain noise should be preserved for the sake of natural look, however, it tends to reduce the coding efficiency because of its random nature. Then a natural idea to overcome this problem is to remove film grain noise as much as possible prior to video encoding. Here, we estimate the essential parameters of film grain noise and adapt the GMHMCF (or CIFIC) for noise reduction. Then at the decoder size, to reproduce the natural appearance, film grain noise is modeled by an auto-regressive (AR) model, synthesized using the estimated parameters, and added back to the decoded video. Simulation results show that the proposed scheme can considerably improve the coding efficiency and at the same time achieve satisfactory subjective quality.