Study On Fast Mode Selection Algorithm For The Coding Standard H.264/AVC

H.264/AVC is the latest international video coding standard developed by both ITU-T and ISO/IEC committees. With high compression and good network adaptability compared with prior standards it has been widely used in video communication field and video storage broadcast domain. However, it also leads to high computational complexity at the encoder mainly due to the fact that the H.264 encoder employs more complicated approaches and features to improve the coding performance efficiently, this can't meet the real-time application of H.264. Hence, it is an essential research topic to reduce the high encoding complexity and the encoding time while maintaining the good coding performance for the convenience of the application of H.264 in real-time system.After researches of the key techniques and the complexity of H.264/AVC video coding in detail, the paper focus on optimization for the mode selection algorithm about inter prediction and intra prediction. JM8.6 reference software was used for the testing model in the experiment, and the coding programme was optimized, then procedures were debugged with Visual studio6.0 to testify its advantage of encoding effect. The paper has completed the following aspects of the research work:①Based on the analysis of coding theory and intra-frame mode selection algorithm of the JM8.6 reference model, the question of complexity has been pointed out firstly, then we compared all kinds of intra mode selection algorithm from different angles, and finally the intra-frame mode selection algorithm was optimized. The edge was divided into five categories in this algorithm, then the block was divided into four small blocks, and the key edge strength of the block was calculated according to edge filter matrix, then the most likely candidates prediction model of this block were gained in line with the key edge strength. At the same time we composed the most likely prediction model set of intra prediction combined the adjacent prediction model.Finally we gained the final encoding mode of the block using the rate-distortion cost function. Experimental results demonstrated that the new algorithm lowers the complexity of the algorithm and reduces the encoding time without obvious PSNR change.②The paper pointed out the huge computational complexity of inter block prediction mode selection algorithm after the research about the inter-frame coding and inter-frame block division mode selection algorithm process in H.264/AVC.In the light of huge computational complexity of inter block prediction mode selection algorithm in H.264/AVC, the paper made improvement on the original inter frame mode selection algorithm. The skip mode was gived the highest priority level in the improved algorithm, and it was given a quick judge firstly, then it determines whether it is need to do intra prediction in the inter prediction according to the image feature, finally we choose the best block partition mode in line with the relation between the sum of absolute difference and the mean sum of absolute difference. Experimental results showed that the new algorithm could improve the coding speed greatly with little change on PSNR and bit rate compared with traditional full search algorithm.