Studies On Rate-Distortion Optimization Algorithm In Image Coding

The starting point of classical rate distortion theory can be found in Shannon's paper"a mathematical theory in communication"in 1948. Rate-distortion theory comes under the umbrella of source coding or compression, which is concerned with the task of maximally stripping redundancy from a source, subject to fidelity criterion. Rate-distortion optimization greatly improves performance of coding system so that it pervades all of source coding, both from an information-theoretic standpoint as wll as for the design of practical coding systems. For many years rate-distortion optimization has attracted more and more researchers. In most lossy compression system quantizer greatly impacts the coding performance. With an increasing demand for efficiently compressed representation of imiages and video signals, the task of optimial quantization has assumed renewed importance. The research of this thesis focuses on rate-distortion optimization technology and how to improve quantizer by rate-distortion optimization in image and video compression. The content of this thesis is introduced as follows:Firstly, we investigate how to pick the optimal slope valueλ~* of convex hull of the rate-distortion curve for a given budget criterion Rc. Aiming at the drawback of bisection algorithm, we proposed golden-ratio algorithm to find out the optimal slope for budget criterion Rc. Firstly the biased Lagrangian cost W(λ) which is a concave function ofλis introduced. Then it can be proven that the value maximizing W(λ) are the optimal convex full face slopeλ~*. Finally golden-ratio algorithm is proposed to search the value that maximizes W(λ), that is, the optimal slope valueλ~*. Starting from a known initial interval engulfing the desired operating slope, the search intervals are made successively smaller until convergence is achieved. In addition, the convergent rule of Golden-ratio search algortihm is improved to get faster convergence rate. The algorithm is sample and easy to be realized. Moreover, it converges in any situation. The closer the initial search window is to the final value, the fewer iterations are needed. The experimental results show its iterative times can be even less than Bisection's. In addition, the proposed algorithm can improve the coding performance. A gain about 0.6-0.7 dB can be achieved with the same rate in H.264.Secondly, how to improve the quantization performance in block-based image and video compression is investigated. A rate-distortion based quantization level adjustment (RDQLA) algorithm is presented. Considering the distribution characteristic of the input signal of the quantizer, the proposed algorithm focuses on the small signals. Based on rate-distortion criterion, the quantization level adjustment algorithm optimizes quantization levels of the signals near the boundaries of deadzone. Except that RDQLA algorithm also makes improvement on block coding process. The sparse non-zero coefficients occur in a block after transformation, which is not worth coding from the viewpoint of rate-distortion. In order to avoid this case, a rate-distortion optimized block coding method is used. The proposed algorithm has no overhead and is fully compatible with the existing compression standards. It can be applied in any block based image and video coding method. In particular, the algorithm has been verified on the platform of H.264 and H.263. Experimental results show that RDQLA algorithm improves both objective and subjective performance substantially. The RDQLA algorithm substantially surpasses the original algorithm, with benefits becoming significant in the case of high bit rates. For QCIF sequence"news"the gains is about 0.2dB at the low rate of 40Kbit/s and is more than 2dB at rate of 1.3Mbit/s。When the quantization parameter is very small, like 4, the gains even exceeds 5 dB.Finally, we investigate how to optimize the trellis-coded quantizer. Based on the analysis to the trellis-coded quantizer, a rate-distortion optimized Trellis-Coded Quantization (RDOTCQ) algorithm is presented. The proposed algorithm has two steps which respectively aim at the two steps in trellis-coded quantization. Firstly, the selection of the reproduction signal in each subset is optimized according to rate-distortion theory and then the trellis path selection is optimized by introduces the rate-distortion cost into the cumulative path transfer cost. Based on rate-distortion criterion, the proposed algorithm effectively improves coding efficiency. In addition, it has no overhead and is fully compatible with the standard decoder. The proposed algorithm can be applied in any TCQ-based systems. In particular, the algorithm has been verified on the platform of the quadtree classified and trellis coded quantized (QTCQ) wavelet image compression system. Experimental results show that the proposed algorithm has a better rate-distortion performance than the QTCQ and other compression methods. On the average, RDOTCQ surpasses QTCQ about 0.2 dB. A maximum gain up to 1.0 dB can be achieved compared with SPIHT.