Compression Domain Based Image/video Interpolation And Reconstruction Technology

Recently,the digitized media technology has been widely applied in many fields for various applications are developed constantly.Among all kinds of digital multimedia signals,the characteristics of visual perception o f image/video makes it becomes the main information carrier of the human visual signals.There are some problems on the compressed image/video.On one hand,due to the large source data of image/video,most images/videos are transformed into bitstreams by lossy coding technology for the efficient transmission and storage.Because of the limitation of the acquisition device resolution or the limitation of the storage space and transmission bandwidth,some image/video signals are with low resolution.On the other hand,by using the compressive sensing coding technology,some image/video signals are with low reconstruction quality.The image/video with low resolution or low reconstruction quality impacts the visual perception of users.Therefore,the interpolation and reconstruction technology for compressed image/video is one of these image processing topics with actual meaningful.The reconstructed image/video decoded by the bitstream can be seen as the degradation of the original image/video signal,leading to the ill-posed reconstruction problem.This problem involves: the video coding technology and the image processing.The side information to guide the image/video reconstruction in the bitstream can effective estimate of the degradation process of the original image/video signal.According to the characteristics of image/video,the priori model of natural image/video has also been introduced on the reconstruction signals to constrain the solution space of the original image/video and enable the inverse problem well-posed.In conclusion,by utilizing the side information of the bitstream and the priori model of image/video,the interpolated or reconstructed image/video can satisfy the human visual perception.Therefore,the interpolation and reconstruction technology for compressed image/video is one of these image processing topics with theoretical significance.In this thesis,based on the side information in the coding bitstreams and the image/video prior modeling,we mainly focus on the research topics on the interpolation for compressed image/video and the reconstruction for compressive sensing image/video.The contents are divided into four sections as follows:First,we propose a local-nonlocal model based compressed image interpolation.By analyzing of the side information in the bitstreams,the confidence interval for the coefficients in transform domain is obtained,and the soft data accuracy in transform domain is proposed for compressed images.By considering the local and nonlocal characteristics of image,we integrate the two complementary image prior model: the local autoregressive model and the nonlocal adaptive 3-D sparse model.Estimating the high-resolution image by the local AR regularization is different from these conventional AR models,which weighted calculates the interpolation coefficients without considering the rough structural similarity between the low-resolution and high-resolution images.Then the nonlocal adaptive 3D sparse model is formulated to describe the sparsity of these similar image blocks.Combined with the proposed soft data accuracy and local-nonlocal image prior model,we present the regularization framework of compressed image interpolation.In addition,a new Split-Bregman based iterative algorithm is developed to solve the above optimization problem iteratively.Experimental results show that our proposed method outperform the compared methods in both subjective and objective results.Second,a directional frame interpolation for compressed video is proposed.The existed image interpolation methods may not be efficient when directly applied to compressed images or videos,because they do not utilize the information existed in the bitstreams,such as block direction,motion vector and quantization.Inspired by the success of the intra prediction in video coding and the edge-directed image interpolation methods,a directional frame interpolation for compressed vi deo(the intra frames and the inter frames)is proposed.This compressed video interpolation framework utilizes both the spatial and the temporal side information in the bitstreams.Specifically,for intra frames(I frame),the intra prediction direction information is taken into account as side information in the directional interpolation.For each intra block,the interpolation is performed along its block direction.The interpolation weight for each block direction is off-line trained by the Wiener filter based on the representative video sequences.In the similar way,for each pixel in an inter block in P or B frames,the interpolation is performed along the direction of its corresponding reference blocks indicated by the motion vectors.Third,a novel local structural measurement matrix(LSMM)for block-based compressive sensing(CS)reconstruction of natural images is proposed.Gaussian random matrix(GRM)has been widely used to generate linear measurements in compressive sensing of natural images.However,in practice,there actually exist two problems with GRM.One is that GRM is non-sparse and complicated,leading to high computational complexity and high difficulty in hardware implementation.The other is that regardless of the characteristics of si gnal the measurements generated by GRM are also random,which results in low efficiency of compression coding.In this paper,we design a novel local structural measurement matrix for block-based CS coding of natural images by utilizing the local smooth pr operty of images.The proposed LSMM has two main advantages.First,LSMM is a highly sparse matrix,which can be easily implemented in hardware,and its reconstruction performance is even superior to GRM at low CS sampling subrate.Second,the adjacent measurement elements generated by LSMM have high correlation,which can be exploited to greatly improve the coding efficiency.Furthermore,this paper presents a new framework with LSMM for block-based CS coding of natural images,including measurement generating,measurement coding and CS reconstruction.By analyzing of the side information on the decoder,the confidence interval for the coefficients in measurement domain is obtained,and the soft data accuracy in measurement domain is proposed for compressive sensing images,which outperform the traditional reconstruction method at the low bitrate.Experimental results show that our proposed method outperform the compared methods.Fourth,the spatial-temporal model based hierarchical compressive sensing video reconstruction is proposed,in which the whole video is independently divided into non-overlapped block based hierarchical frames with different sample subrates.The proposed framework outperforms the traditional framework through the better exploitation of frames correlation with reference frames,the unequal sample subrates setting among frames in different layers and the reduction of the error propagation.Each measurement of the block in different hierarchical frame is obtained with the different sample subrate.At the decoder,by considering the spatial and temporal correlations of the video sequence,a spatial-temporal sparse representation based reconstruction of the compressive sensing video is proposed,in which the similar blocks in the current frame and these recovered reference frames are organized as a spatial-temporal group unit to be represented sparsely.By analyzing of the confidence interval for the coefficients,we propose the soft data accuracy in measurement domain.Combined with the proposed soft data accuracy and spatial-temporal sparse model,we present the regularization framework of the compressive sensing video reconstruction.Finally,the problem can be solved by adopting the Split-Bregman iteration.Experimental results show that our proposed method outperform the compared methods in objective results.Therefore,based on the side information in the coding bitstreams and the image/video prior modeling,we proposed the interpolation methods for compressed image/video and the reconstruction methods for compressive sensing image/video.Experimental results show that our proposed method outperform the compared methods in both subjective and objective results.