Research On Rate-Distortion Optimization Technology And Its Application In Video Coding

As the conventional forms of visual signal,image and video data have been of higher resolution and dimension,which provides a higher quality of viewing experience.Furthermore,with the development of artificial intelligence,human beings is desiring to live a much more intellectual life.However,data-driven applications require a huge volume of image and video data,which puts a tremendous pressure on cost-effective storage and transmission and further blocks the development of multimedia applications.To emerge the desires and improve the coding performance,international standard organizations have developed several coding standards for image and video compression.Unfortunately,the coding performance is still not good enough,especially for high and ultra-high definition videos as well as a few 3D sources.Most of coding technologies are investigated to further improve the coding efficiency at the cost of increasing the coding complexity,which cannot make the codec practical in real-time applications.Meanwhile,most of codecs are developed for traditional image and video data formats,which cannot compress the other kind of data efficiently,such as light field image data.Finally,lossy compression scheme is commonly used to achieve a better coding performance,which results in an inevitable compression error and then puts a negative influence onto video analysis tasks in multimedia applications.This dissertation aims at the abovementioned issues from the angle of rate-distortion optimization.The main contributions are summarized as follows.1.A better coding performance can be achieved by providing more transform options.However,the coding complexity also increases a lot along with the number of transform options.To make a better trade-off between the coding performance and coding complexity,this dissertation formulates the relationship among intra mode decision,the number of transform cores and coding complexity,and proposes dual neighboring commutative mechanism by considering the new characteristics of intra predictions techniques with a small angle between two neighboring intra prediction angles.According to the proposed mechanism,different horizontal and vertical transform cores are assigned to two neighboring intra modes.Meanwhile,following the idea of mode-dependent transform,new mode-dependent transform tables are designed according to the statistical results of intra prediction residuals.The designed transform tables are further modified according to the dual neighboring commutative mechanism.Experimental results show that Adaptive Multi-core Transform can achieve about 3.38%,0.69% and 0.75% BD-rate savings on luma and chroma components with 78% encoding time increase.The proposed method can achieve about 2.15%,2.55% and 2.60% BD-rate savings on luma and chroma components compared with JEM-7.0.Especially on 4K sequences,the proposed method can achieve up to 3.97%,3.75% and 4.59% BD-rate savings on luma and chroma components,respectively.The encoding time of the proposed method increases only 6% compared with JEM-7.0,which indicates that the proposed method can make a better trade-off between the coding efficiency and the coding complexity.2.In order to omprove the coding efficiency of light field image,this dissertation follows the idea of pseudo-sequence based coding methods,where light field image is firstly converted into sub-view images and then a pseudo-temporal sequence is generated according to a pre-defined scanning order and finally compressed by a conventional video codec.It should be pointed out that scanning order is important in the pseudo-sequence based coding scheme.Considering the temporal dependency between the coding frames,this dissertation proposes a new scanning order in view of the disparity and visual quality difference between sub-views,which makes the correlation between two adjacent frames in the generated pseudo-sequence stronger than that in the other kinds of scanning order.Experimental results show that the proposed method can achieve about 16.4% BD-rate savings compared with the zigzag order based coding method.Meanwhile,in order to improve the coding performance,temporal-dependent rate-distortion optimization method is used in the coding processing and a QP adaptation method for I frame is proposed as well.Experimental results show that the improved method can achieve about 21.8% BD-rate savings compared with the zigzag order based coding method.3.Compression distortion would put a negative impact on object detection,which makes the detection results different between the original video and the compressed video.This dissertation first utilizes the term “analytical distortion” to describe the differences of detection results and then analytical distortion is introduced into the rate-distortion optimization process to minimize the sum of the compression distortion and analytical distortion under a constrained bit budget.In order to refrain from multi-pass coding process to obtain the compression distortion and analytical distortion,this dissertation proposes a two-pass coding method,where the analytical distortion is predicted by compression distortion which can be obtained after the first coding process.Furthermore,a rate-analytical distortion model is built according to the statistical results of analytical distortion under different rates.Finally,the optimization problem can be solved by using a relaxed Lagrangian multiplier method.In order to further reduce the coding complexity,a one-pass coding method is proposed,where the compression distortion is estimated according to the temporal reference relationship.Experimental results show that both of two proposed methods can achieve up to 40% analytical distortion reduction compared with HEVC(High Efficient Video Coding).