Research On Texture And JND Modeling Based Video Coding

The current typical video coding standards are based on prediction/transform hybrid coding framework, which was formulated based on the Shannon-Fano information theory. The most widely used compression principle is still at the level of digital signal processing, mainly from the removal of data redundancy. The compression efficiency mainly relies on the computing complexity increase at the expense of the technical details of fine-tuning. Currently, the video coding technology based on Shannon-Fano information theory is facing a break bottleneck. The way of increasing computational complexity to improve the compression efficiency is becoming invalid. How to improve the compression efficiency is an important problem. Perceptual video coding, which is based on human visual system theory, can greatly reduce bit rate while maintaining the same effect of subjective perception. It is of great significance for the solution to this problem.This dissertation first analyzes the typical representative of perceptual video coding: texture synthesis and JND model, concentrates on analyzing the research status of video temporal prediction, implicit motion prediction and JND based video coding technologies, and draws a conclusion:Because the research on the mechanism of human understanding should be further studied, the progress of video coding techniques entirely from the human visual system is still slow, and there are many deficiencies need to be solved. However, based on the local visual characteristics, a few researchers introduce some perceptual coding tools into the traditional video coding framework. This method can break through the traditional compression ideas, which rely on increasing the computational complexity to improve the coding efficiency.Based on the above analysis, this dissertation researches on video coding based on the texture and JND modeling, which is supported by the National Natural Science Foundation of China "A research of video coding based on texture model" (No.61003184), the National Natural Science Foundation of China "A research of video coding based on inverse texture synthesis" (No.60970160) and the Microsoft Research Asia Project based Funding "Improved JND model based on contourlet transform and image structural information" (No. FY09-RES-OPP-013). Based on texture and JND models, this dissertation establishes a video coding framework which is compatible for the traditional hybrid video coding framework. By using the perceptual coding techniques as a tool set, it breaks through the bottleneck in traditional video coding techniques and improves the efficiency of video coding, which has a strong theoretical value. And the work is very significant for the high definition video applications on demand to enhance compression efficiency and the wireless broadband mobile applications to enhance the fault tolerance performance.The major contributions of this dissertation are as follows:(1) The video encoding and decoding algorithm based on dynamic texture modelThe traditional method for solving the dynamic texture model uses the average of the previous frames as the reference value of synthetic image, making the synthesis frame to be characterized by a period of time trend of overall image. This method is not suitable for the inter prediction of video coding. Some researchers have proposed an improved method and omitted the noise item in the original dynamic texture model to ensure the encoder and decoder data matching. However, in the signal processing and system theory, the image sequences can be thought of as the consequence of a bivariate stochastic process driven by the noise item. If the noise item is omitted, the dynamic texture model would not be driven in theory. To solve this problem, this dissertation gives an improved solution for the dynamic texture model. By using the pseudo-random number to describe the noise item, this method would make the synthetic frame has a smaller synthesis error. Based on the improved method, this dissertation presents a new algorithm for dynamic texture extrapolation using for H.264 encoding and decoding system. The synthesized frames can be used by the encoder for virtual reference frames choice in inter prediction which might improve the inter prediction on the sequence with non-linear motion and global illumination change between frames. And the synthesized frames can be also integrated into the decoder for whole frames loss error concealment, which achieves significant improvement over the traditional motion vector extrapolation method.(2) The inter prediction algorithm based on STALL modelThe original Spatio-Temporal Adaptive Localized Learning model is implemented in a pixel-wise fashion. For each pixel, the scheme identifies its spatial neighbors as well as its temporal neighbors within a causal window. But for the lossy compression, for example, H.264 standard adopts 4X4 block transform structure and the reconstructed pixels are not identical to the original pixels, the spatial neighbors in the 4×4 block could not be accessed when the 4×4 block hasn't been encoded yet. To solve this problem, this dissertation designs an improved STALL model by proposing an adaptive spatial and temporal neighbors' selection strategy, and adds an LSP inter prediction mode into H.264 standard for lossy compression which improves the accuracy of inter prediction.(3) The adaptive residue filter algorithm based on color JND modelThe previous color JND models are usually based on RGB or YCbCr color spaces. Because these color spaces are not uniform color system, the perceived color change produced by a fixed small change of the color coordinates is non-uniform. It would be a problem when computing the chrominance JND component and thus the precision of chroma JND threshold value should be improved. To solve this problem, this dissertation introduces a new color JND model based on the CIELAB color space, which the value of chroma JND component holds with good precision. And then, an adaptive residue filtering algorithm would be proposed, which can increase the compression efficiency of H.264 standard while having the same subjective perceptual quality.In conclusion, this dissertation researches on the video coding based on texture synthesis and JND model, establishes a compatible hybrid video coding framework and proposes a series of video coding algorithms, which could improve the video compression efficiency. It is of significance for research and development of video coding and communication systems. Lastly, the dissertation summarizes the research achievements and looks into the future research in the area of multi-view video coding and video quality assessment. Based on the achieved research achievements, we expect to do further exploration on the texture synthesis based multi-view video coding and perceptual video quality assessment.