A Study On Transform And Quantization Based On H.264/AVC And Its VLSI Implementation

H.264 is a new video compression standard being jointly developed by ITU-T Video Coding Experts Group and ISO/IEC Motion Picture Experts Group. Because it adopts so many new techniques that it has achieved up to 50% in bit rate saving compared with H.263 and MPEG-4 considering the same rebuilding image quality. The layered coding thought is introduced in H.264. The key techniques include intra-prediction, inter-prediction, high-precision and multiple reference frames for motion estimation, 4×4 integer transform and quantization, deblocking loop filter, context adaptive variable length coding(CAVLC) and context adaptive binary arithmetic coding(CABAC).The principle of the transform and quantization in H.264 is researched in this thesis. The new transforms adopted in H.264 can be computed exactly in integer arithmetic and without multiplication, just additions and shifts, in 16-bit arithmetic, thus minimizing computational complexity and making the hardware implementation easier. Meanwhile, the inverse transform mismatch problems can be avoided. It also can reduce the blocking artifacts, the ringing artifacts and the mismatch problem in the inverse transform process. It makes H.264 an effective video coding standard by using the 4×4 transforms. These changes lead to a significant complexity reduction and have guaranteed the precision, with an impact in peak signal-to-noise ratio (PSNR) of less than 0.02 dB.A fast algorithm called papilionaceous operation is adopted to achieve the hardware implementation in this thesis. Papilionaceous operation can avoid many useless calculations and can achieve a high speed in computation. Meanwhile, because the 2-D integer transform can be divided into two 1-D transforms, every 1-D transform can be implemented using papilionaceous operation. The parallel architecture is often used in the integer transform module, although it can achieve a high operation speed, it has many disadvantages, such as the bigger area. This design has an improvement in the hardware architecture: the transforms of the column and row are divided, first, a column of the residual 4×4 matrix is transformed using papilionaceous operation, then every row of the...