Rate Distortion Optimization Based Video Coding

As the original video data is so voluminous for real applications, video coding has been a focus of research and applications for saving transport bandwidth and storage space. For now, many video coding standards have been established for different applications, such as MPEG-1/2/4, H.261/H.263. In the latest, the just released H.264 and AVS standards have achieved significant improvement compared with previous video coding standards, but they have similar framework! So it is very important for us to have an in-depth study on the key techniques used in video coding and do some improvements based on the coding scheme. In this paper, rate distortion optimization coding and rate control are considered together. Based on the analysis on the transform and quatization scheme, optimal bit allocation at coefficient level is reached for a block though adaptive deadzone quantization. For macroblock and picture level bit allocation, a rate distortion optimization based rate control algorithm is proposed under the buffer constraint. The main contributions in this paper are listed as follows:1. Rate distortion optimization based optimal bit allocation for transform coefficients In this paper, based on the in-depth study for the current transform/quantization scheme, an optimal bit allocation at coefficient level is realized by adjusting the deadzone length for each block coefficient. With the adaptive quantization scheme, both coding efficiency and visual quality can be improved. In the latest H.264, AVS standard, transform and quantization have shown some new features. First the float DCT transform has been replaced with integer transform and quantization is combined together with transform normalization. Especially, in AVS, a transform matrix with similar norm for every row is used and the transform normalization can be implemented only on the encoder set.That would affect the optimal coefficient level bit allocation. Based on the characteristics of the new transform/quantization scheme, an optimal bit allocation scheme at coefficient level is proposed.2. Buffer constrained rate controlIn this paper, based on a detailed analysis on H.264 HRD model, a buffer constrained rate control algorithm is developed. In the rate control process, the bits allocated to a picture are adjusted based on the buffer status and the buffer status can be guaranteed to be never...