Research On Key Technologies Of Image And Video Coding Based On Multi-model Joint Optimization

With the rapid development of applications such as 4K live broadcast,remote con-ference and surveillance video system,and the explosive growth of image and video data,the efficient storage and transmission of massive image and video data bring un-precedented challenges to coding and compression technology.Therefore,exploring more efficient image and video compression technology to further improve the effi-ciency of compression is the fundamental goal of image and video coding.The coding problem is essentially a rate distortion optimization problem.The traditional coding method mainly uses the hybrid coding framework.Based on the relevant experience of image processing and computer vision,this method removes the information redun-dancy in the video image step by step by manually designing algorithms such as predic-tive coding algorithm,transform coding algorithm,and entropy coding algorithm,so as to achieve compression.At each step,the rate distortion optimization problem is solved by selecting the best mode from several different coding modes.However,this method has two problems.On the one hand,it highly relies on artificial design algorithm and manual parameter tuning.Due to the limitations of artificial experience,the efficiency of the algorithm is often low in some complex or special scenarios.On the other hand,the optimization of different modules is independent,and the interaction between dif-ferent modules is not considered.Theoretically,the rate distortion optimization will be in the local optimum.Aiming at the problems existing in the application of multiple models in the past coding framework,this thesis first studies the accurate model design and multiple mod-els combination optimization algorithm in traditional coding framework;then combines the end-to-end compression method with traditional framework to study the end-to-end compression algorithm based on ensemble learning;finally,the cost of multi-model coding is introduced into multi-model training to study the multi-model rate distortion joint optimization.The main research work and contributions are as follows:(1)Aiming at the problem that the accuracy of multiple models in traditional cod-ing framework are poor,this thesis studies the accurate model design and multi-model combination optimization algorithm for special scenes.Therefore it studies the com-plex motion deformation in panoramic video.It first designs an inter prediction motion model based on spherical coordinate transform from the perspective of mathematical theory,which can better describe the motion deformation in panoramic video.Then the new inter frame coding algorithm is designed,including motion compensation al-gorithm and motion estimation algorithm,and some ingenious methods are used to sim-plify the algorithm and integrate it with the existing coding framework.Finally,the new model is compatible with various motion models in the existing framework to realize the harmonization of multiple motion models and fast optimization algorithm.Experiments show that the new motion model and multi-model combination optimization method can effectively improve the inter prediction accuracy,improve the compression efficiency,and reduce the decoding complexity of panoramic video.(2)Aiming at the problem of joint optimization between modules in the traditional coding framework,this paper attempts to break through the traditional framework and combine the end-to-end network compression method with the traditional framework,and with the help of the idea of rate distortion optimization method of multi-model opti-mization in the traditional framework,to solve the problems of poor adaptation and high complexity of the end-to-end compression network model,so as to propose an end-to-end image compression framework based on ensemble learning.The framework adopts block adaptive model selection,and several model generation algorithms for models are designed in the framework,and uses model generation methods such as modified boost-ing and Geometric Self-ensemble to solve the problem of high training cost and ensure the diversity of models.The experiments show that the end-to-end image compression method based on ensemble learning can effectively improve the compression efficiency without increasing the decoding complexity.Conversely,it can reduce the decoding complexity while ensuring the compression efficiency.Furthermore,The experiments prove that our method has good generalization ability.(3)Aiming at the problem that multi-model coding cost is not considered in multi-model training,this thesis attempts to introduce model coding rate cost into multi-model training,and proposes a joint optimization method of multi-model rate distortion,and the application scene is oriented to deep neural network image loop filtering.Firstly,a multi-model joint training method is used to train multiple filter networks,and then a method to control the coding rate of the model under different rate constraints is de-signed.Then through the training method based on annealing,the collapse problem in model training is solved.Furthermore,this thesis design a two-step model selection method with model group selection based on distortion amplitude and model selection,which achieves better rate distortion performance than only using model selection.At the decoding side,the whole framework of deep neural network loop filtering is con-structed through the image block level two-step adaptive model selection.Experimental results show that this method can effectively control the coding rate of multiple model,suppress model collapse and improve the rate distortion performance of multi-model deep neural network filtering.