H.264 Mode Selection Algorithm Research And Its DSP Implementation

H.264/AVC is the latest video compression standard enacted by ITU-T and ISO/IEC Joint Working Group. By adopting a number of new coding techniques, it has achieved a significant improvement in compression performance and a "network-friendly" video representation relative to existing standards. In addition, DSP technology, which develops so quickly recently, could provide a platform for the complicated new generation video coding system. Consequently, research on H.264/AVC video coding algorithms and its realization with DSP has become a hot and difficult subject in video communication industry.Research achievements on Algorithms: Rate distortion optimized strategy, intra prediction, variable-sized block inter prediction mode selection, integer core transformation and quantization, and in-the-loop deblocking filtering were studied deeply by the reference JM8.6 code in this paper. H.264 adopts variable block size Motion Estimation (ME) and exhaustive RDO mode decision. The encoder finds the best coding mode of a macroblock by computing the RDO of ten modes. So it remarkably increases the complexity of the encoder. A fast mode decision algorithm base on motion complexity is presented to improve the efficiency of the mode decision. Firstly, the algorithm divides video area into three types bath on the motion complexity of video frequency. Then calculate the motion complexity of the area. It can reduce the search modes by using time and spatial relativity. The experiments results showed that, compared with the original full research method, this fast method could reduce inter encoding time by 20%～26% with little influence on encoding performance.Research achievements on DSP realization: This dissertation tries to probe the coding principle of H.264/AVC, and optimize the reference codec on its basis. The main research and results are listed below. By using the way of system optimization, algorithm optimization and instruction optimization, making the best of parallelism disposal capability and abundant of media process instruction. Experimental results show that it substantially improves the encoding speed with QCIF format video on the Blackfin533 DSP.