Study On The Key Technology In Video Denoising

Video noise reduction technology can be used not only to filter out the noise in video, which enhances the visual quality, but also to enhance the performance of the subsequent processing tasks such as compression, target recognition and tracking, as well as frame interpolation. The existed video denoising algorithms can be classified into two categories: the earlier pixel-domain methods and transform-domain methods which appear in recent years.Based on the region of filtering, pixel-domain noise reduction methods can be classified into temporal filtering and spatiotemporal filtering methods, respectively. Temporal methods suppress noise in video by utilizing temporal correlation, which is often abtained through motion estimation/motion detection (ME/MC). And spatiotemporal methods utilize three dimensional correlation to suppress noise. A disadvantage of pixel-domain methods is that artifacts will be introduced into the denoised video sequence by temporal filtering or spatiotemporal filtering. So, there is no one filter that can be singled out as the solution for all applications. But an interesting trend observed is the importance of adaptability, which will reduce the effects. Another problem with pixel-domain methods which are combined with motion compensation in most cases is that the presence of noise will affect the accuracy of motion estimation, which will deteriorate the denoising performance accordingly. Thereore, there is an urgent demand on the motion estimation algorithm which is suitable in noise environment and robust to noise.Wavelet analysis is a classical signal analytical tool which is widely used in various signal processing areas. Because of the multi-scales decomposition capability, two dimensional wavelet transform and three dimensional wavelet transform have become new focus in video denosing technology in recent years. There are a variety of methods in multiscale geometric analysis transform domain, such as the methods of energy model and inter-scale correlation model which are based on physics models, and methods based on statistical models like the hidden Markov tree model and inverse Gaussian distributions model. But the methods based on physics models study the energy or values of video and noise coefficient which sometimes can not be consistent with the video detail well, so that video detail is damaged. And the correspondence relationship between the statistical models and video details still needs to be studied, wherein lie many parameters and priori limitations on their applications. In brief, denoising method in multiscale geometric analysis transform domain still needs to be studied, and the key problem is to find out more suitable video denoising models, parameters and algorithms.The main objective of video denoising is to filter out the random noise, while maintaining details as well as possible and reducing artifacts introduced by the filtering. Based on the above aim and noise suppressing, the recearch involved in this thesis includes motion estimation and motion compensaton, cross bilateral filtering, two dimensional wavelet transform, three dimensional wavelet transform, adaptive mechanism, etc. And the protection of the video details is the excellency of the thesis.The main works of this thesis are:①Down-sampling block matching fast motion estimation algorithm.A down-sampling block matching fast motion estimation algorithm is proposed combining multi-resolution analysis motion estimation theory. In this algorithm, each frame in video sequence will first be decomposed according to the two dimensional sparse representations in the first step. Then initial motion vector field will be obtained from the low resolution layer. At last in the corresponding higher resolution layer, a more sophisticated motion vector field will be got. Experiment results show that: The speed of this algorithm is about 1.26~1.8 times as fast as that of traditional fast search methods, and the searching accuracy is about 4%~38% better than that of the traditional method. The search feature of this algorithm is robust to different levels of noise. All of the results show that this fast search algorithm is suit for video denoising application.In the down-sampling block matching fast motion estimation algorithm, on one hand fast searching speed can be achieved in the premise of searching accuracy by using bilinear interpolation technique in two resolution decomposition, on the other hand, in noisy cases accurate motion vector field estimate can be achieved due to the low-pass characteristics with bilinear interpolation down-sampling.②With the introducing of adaptive mechanism, a multihypothesis recursive spatiotemporal filtering (MRF) video denosing algorithm based on motion state detection is presented.In noisy video sequence, current frame will be divided into non-overlapping blocks of equal size and its motion state will be detected combining multihypothesis motion estimation first. Then, based on the local motion state, different denoising schemes will be selected to suppress noise. Areas with stationary motion state will be filtered using temporal filtering, whereas those with non-stationary state are filtered using self-cross-bilateral filter (SCBF). The definitions of stationary motion state and non-stationary motion state are given. To detect local motion state accurately, a threshold equal to the noise standard deviation is assigned. Experiment results show that: With this algorithm, the noise in degraded video will be effectively smoothed away while details are preserved well due to the reasonable utilizing of spatiotemporal correlation between consecutive frames. MRF outperforms conventional denoising methods like joint filtering scheme (JNT), spatio-temporal varying filter (STVF) and multihypothesis motion compensated filter (MHMCF) both in peak signal-to-noise ratio (PSNR )and visual quality.As a result of adaptive mechanisms, MRF could suppress noise effectively by reasonably exploiting temporal and spatial correlation and adopting corresponding filtering schemes. The algorithm model shows a clear geometric meaning and no derogation such as blocking effects would be introduced.into the denoised video.③An improved video denoising method based on motion compensated sphere bilateral filtering (MCSBF).For video sequence degraded by noise, reference frames of the current frame are built first by motion compensation between the current frame and the past/future frames. Then in the three dimensional space composed of the current frame and the compensated reference frames, a three dimensional bilateral filter in which the filtering window is a sphere is used to suppress the noise in the current frame. By fully utilizing temporal and spatial correlations in video content, the proposed method can effectively suppress the noise in the video while keep textures and details well. Experiment results show that MCSBF outperforms conventional denoising methods like JNT, STVF and MHMCF both in PSNR and visual quality.Combined with motion compensation, MCSBF can fully utilize temporal and spatial correlations in video content to suppress noise while keep details well. The reasons are: Temporal non-stationarity of video content can be removed by motion compensation (from a macro point of perspective), and spatiotemporal non-stationarity can be removed by the Gaussian kernel of radiometric distance parameter (from a micro perspective). Textures and details can be preserved well in the denoised video for that pixels involved in filtering are all high correlated with the central filtered pixel. And the three dimensional sphere filtering window can effectively reduce the number of pixels with low correlation, which will introduce over smoothing artifacts. Denoising performance of MCSBF will not be affected by local rotation, because the filtering window centered at the filtered pixel is with the feature of rotation invariance and can adapt to the local rotation phenomenon; Pixel based filtering scheme can avoid introducing blocking artifacts into the denoised video from the essence.④By combining wavelet thresholding and cross bilateral filtering, a motion compensated 3D self-cross bilateral filtering video denoising filter (3DSCBF) based on undecimated wavelet transform thresholding method is proposed.For video sequence degraded by additive Gaussian white noise, motion compensated noisy bilateral consecutive frames of current frame will be obtained first. Then these frames and current frame are denoised utilizing undecimated wavelet thresholding method. At last, noisy pixels in current frame will be denoised by cross bilateral filtering in a sliding 3D window, combining motion compensated noisy frames, current frame and the thresholded frames. The geometric distance parameter is obtained from current frame and its motion compensated frames, while the radiometric distance parameter is calculated in the three dimensions composed of thresholded reference frames. Experimental results show that 3DSCBF can effectively filter out noise and maintain textures and details well. At the same time, 3DSCBF outperforms conventional two dimensional wavelet based denoising methods like undecimated wavelet transform thresholding method (UWT Thresholding) and video sequence noise reduction using wavelet-domain and temporal filtering (SEQWT) both in PSNR and visual quality. And the protection of video details is one excellency of the thesis.In this scheme, non-stationarity of video sequence will be removed based on MC, and the radiometric distance parameter, which is calculated with UWT Thresholded noisy frames, will be more reliable than that of with original noisy frames. The final cross bilateral filtering can also keep the merit of original bilateral filtering. As a result, noise in video sequence can be averaged out and edges can be preserved well by fully utilizing both spatial and temporal correlation. In addition, when the 3D filtering window is optimized from a cube to a sphere, which is invariant to local rotation, the denoising performance of 3DSCBF is improved further.⑤Combining motion compensated three dimensional wavelet transform and cross bilateral filtering, a 3D self-cross bilateral filtering video denoising filter based on motion compensated three dimensional wavelet transform thresholding method (SCBF-MC3DWTTH) is proposed. For video sequence degraded by additive Gaussian white noise, motion compensated noisy bilateral consecutive frames of current frame will be obtained first. Then these frames and current frame are denoised utilizing motion compensated three dimensional wavelet transform thresholding method (MC3DWT Thresholding). At last, noisy pixels in current frame will be denoised by cross bilateral filtering in a sliding 3D window, combining motion compensated noisy frames, current frame and the thresholded frames. The geometric distance parameter is obtained from current frame and its motion compensated frames, while the radiometric distance parameter is calculated in the three dimensions composed of thresholded reference frames. Temporal non-stationarity of video sequence will be removed based on MC, and the radiometric distance parameter which is calculated with MC3DWT thresholded noisy frames will be more reliable than that of with original noisy frames. The 3D filtering window is a sphere which is invariant to local rotation and fits for the local non-stationarity. As a result, noise in video sequence can be averaged out and edges can be preserved well by fully utilizing both spatial and temporal correlation. In addition, when the 3D filtering window is optimized from a cube to a sphere, the denoising performance of SCBF-MC3DWTTH is improved further.From three sets of experimental data, it can be seen that: MC3DWT is an effective multi-resolution decomposition tool in video processing. The first set of experimental data shows that random noise in video can be effectively filterded out through MC3DWT Thresholding method. From comparison in the second and the third sets of experimental data, it can be concluded that SCBF-MC3DWTTH can effectively suppress random noise in video and maintain textures as well as details well, while no apparent artifacts are introduced into the denoised video. SCBF-MC3DWTTH outperforms conventional two dimensional wavelet based denoising methods like Gaussiana scale mixtures (GSM), SEQWT and inter-frame statistical modeling of wavelet coefficients (IFSM) in PSNR, and also outperforms three dimensional wavelet based transform domain denoising methods like motion compensated three dimensional wavelet transform thresholding method (MC3DWTn) and temporal discrete cosine transform and spatial hierarchically adapted wavelet transform thresholding method (DCT+DWT). Three dimensional wavelet transform based methods are superior to two dimensional wavelet transform based methods in video denoising, this can be verified clearly from the first and the third sets of experimental data.