Study On The Algorithms Of Coding Optimization In H.264/AVC

H.264/AVC is the latest international video coding standard issued jointly by ITU-T VCEG and ISO/IEC MPEG. Although it is still based on previous prediction/transform hybrid coding frame, its coding efficiency has been raised significantly by introducing recent advanced coding technologies. Compared to the previous video coding standards, H.264/AVC mainly has the following new features: variable block size motion-compensated prediction, multi-frame reference prediction, integer transform, 1/4-pixel-accuracy motion compensation, context-adaptive entropy coding, and in-loop de-blocking filter. As supporting these new functions, the encoding complexity of H.264/AVC is much higher than previous video coding standards. Therefore, it is important to study the video coding technologies, optimize the performance according to the encoding principles and lower the complexity of encoder without losing the encoding quality. The contents of this thesis are centered on the algorithms of coding optimization in H.264/AVC, and they are introduced as follows:Firstly, we investigate how to encode a video sequence optimally under a bit budget. This can be applied to video storage scenarios. In order to get a set of optimal quantization parameters, the dependency between successive frames has to be considered. Frame-level rate-distortion optimization can be used to solve the above problem. Exhaustive iteration algorithm (EIA) is the optimal frame-level rate-distortion optimization algorithm, but the computational complexity is too high to be implemented in real encoding. Then some sub-optimal algorithms are studied, and a frame-level rate-distortion optimization algorithm based on theρ-domain rate-distortion model and the best sub-optimal algorithm (namely GIA) is proposed in this thesis. According to the proposed algorithm, the optimal set of quantization parameters is obtained by choosing the one that has the minimum rate-distortion cost. As a result, great computation is reduced in the proposed algorithm when compared to GIA. Simulation results show that the encoding time of the proposed algorithm is less than one-eighth of one-pass rate-distortion optimization of GIA with almost equivalent coding quality.Secondly, how to control quantization parameter precisely in the frame-level encoding of H.264/AVC is investigated. In the frame-level encoding of H.264/AVC, the encoded bit of every frame has the direct relationship with the frame-level quantization parameter because every macroblock uses the same quantization parameter. The quadratic rate-distortion model in JM10.2 encoding control often causes the mismatch between the encoded bit and the quantization parameter. An algorithm is proposed to adjust the quantization parameter adaptively through theρ-domain rate-distortion model and better performance is achieved at the same time.Finally, we investigate the fast inter mode decision in H.264/AVC encoding. In order to predict more precisely, H.264/AVC supports variable block size motion-compensated prediction. Rate-distortion optimization is used to get the optimal inter mode. The inter mode decision module is the most time-consuming part in H.264/AVC. In order to lower the encoding complexity, a fast inter mode decision algorithm is proposed in this thesis. The two special inter modes SKIP and P8x8 are decided in advance through the analysis of the temporal and spatial information of the macroblock, and computation is greatly reduced in the proposed algorithm.Combining the above adaptive quantization parameter adjusting algorithm and the above fast inter mode decision algorithm, an optimized frame-level encoding scheme of H.264/AVC is proposed in this thesis. Then the proposed encoding scheme is comprehensively compared with the latest JVT-W042 frame-level encoder control scheme. Simulation results show that the proposed optimized encoding scheme performs better than JVT-W042 in low bit-rate scenarios, and the advantage is bigger when bit-rate is lower. Moreover, compared to JVT-W042, the encoding complexity of the proposed scheme is greatly reduced.