Research On Fast Mode Decision With Rate-Distortion Optimization

Nowadays, multimedia communication has become an important part of social life. At the same time, people's requirements on video quality are also getting higher and higher. Due to the large number of data that multimedia and video sequences contain, video signals have to be highly compressed to reduce the redundancy for the demand of the narrow bandwidth, which leads to the decline of video quality. Second generation video compression standards, such as H.264 and AVS, have adopted a strategy of rate distortion optimizing to achieve balance between video quality and bit-rate. RDO does bring high efficiency while at the same time, brings the high-coding complexity.First, The RDO model is introduced. Based on the transform and quantization method of AVS, a new distortion calculation technique in the transform domain is proposed, and used to estimate the distortion in the pixel domain. Secondly, in response to AVS entropy coding features, the main factors of bit-rate using statistical methods are found, base on which, a new model to estimate entropy coding rate through non-zero coefficients is proposed. The experiments show that the distortion measure and rate estimation can make PSNR and bit-rate have a little change.At last, apply the new RD cost calculating technique to the intra and inter mode decision process. a new fast mode decision algorithm with rate-distortion optimization is proposed. the proper initial value of the adaptive coefficient is found through experiments. Experiments show that during the intra and inter mode decision process, The distortion and bit-rate estimated in the transform domain are used to calculate the RD cost achieves about 60% reduced computation time of rate-distortion cost and saves about 20% total encoding time with little change of PSNR and bit-rate.