Research On The R-D Optimization In H.265/HEVC Under The Limited Computational Complexity

With the emergence of the high-definition and ultra-high-definition video, the video compression technology faced up a great challenge. In 2013, MPEG and VCEG jointly launched a new generation of highly efficient video coding standard H.265 / HEVC. Compared to H.264 / AVC standard, the compression performance of H.265/HEVC can be increased by double delivering the same video quality. Many new coding tools in H.265/HEVC effectively improve the coding performance, but inevitably produce excessive computational complexity at the same time. Fast encoding parameter optimization is a key technology under the case of the limited computational complexity. To improve the coding performance as much as possible, fa st optimization algorithm should be implemented under the framework of the rate-distortion(R-D) optimization, and have flexible adaptability to the restrictions of the computational complexity. Therefore, the R-D optimization of CTU mode selection under the limited complexity has been studied. And a complete R-D optimization theoretical framework has been proposed. What's more, an implementation scheme for x265 encoder has been given.For the extremely high complexity of obtaining the R-D cost of the CTU dividing mode, this paper proposes a fast algorithm for predicting the R-D cost of the CU modes. Through deeply analyzing, the R-D cost contains two parts: the header cost and the residuals cost. The coding conditions(QP, CU size) and the temporal-spatial features(the complexity degree of the texture and the moving status) of the video contents are two factors influencing the two parts of the R-D cost. Through a large number of experiments, two relationship models are constructed in each QP and each CU size, one is the model between MV and the header bits, the other is the model between the temporal-spatial features and the residuals cost. With the two relationship models, the R-D cost of each CU mode can be obtained more quickly. To test the performance of the proposed algorithm, a large number of experiments are conducted, and the experimental results show that the consistency rate of the optimal partitioning between the proposed algorithm and the original algorithm in each CU size is between 78% and 90%.Since there are too many candidate dividing modes for CTU, mode selection is always conducted with high complexity. Therefore, a novel algorithm with controllable complexity is proposed to adaptively reduce the candidate modes of the CTU in this paper. First of all, this paper studies the method of assigning the complexity for each frame and each CTU aiming at controlling the whole complexity accurately and improving the coding performance as much as possible. It's more possible that the non-uniform complexity distribution for different frames and CTUs with different features can improve the coding performance. Secondly, the candidate modes of CTU are sorted according to the predicted R-D cost of CU. Finally, the top several modes are chosen to make up of the set of candidate modes at the assigned complexity. The proposed algorithm can not only control the whole complexity accurately, but also maximize the coding performance.