Research On Image And Video Compression For Panorama Applications

With the rapid development of multimedia technology,panorama applications have gradually attracted lots of attention.Different from traditional two dimension(2D)applications,panorama applications can present the whole scene for viewers stereoscopically,and make them choose the viewpoint freely,realizing the natural interaction between the user and the scene.However,panorama applications bring great challenges to the storage,coding and transmission of data while realizing the immersive viewing experience.In order to solve this problem,researchers specially design spherical panorama applications and object panorama applications based on the actual needs of panorama applications.Both of the above panoramic systems can effectively reduce the amount of data that needs to be transmitted in a panorama application.However,new data formats including panorama images and stereo videos are introduced.It is of great research significance to design efficient codecs for coding these new data formats,in order to fully improve the performance of panorama applications.Therefore,aiming at improving the efficiency of panorama applications,this paper studies the panorama picture and stereo video compression methods from the following three aspects: coding of panorama images in spherical panorama applications;coding of stereo videos in object panorama applications;low complexity coding in panorama applications.Firstly,in order to achieve efficient coding of panorama pictures in spherical panorama applications,this paper starts with the properties of the panorama picture itself,and designs a compression autoencoder which is based on dense connection blocks,realizing end-to-end efficient compression of panorama pictures.In the design of the autoencoder,this paper achieves the fully extraction of feature maps,the multi use of feature maps,and feature redundancy reduction by introducing the specially designed dense connection blocks,which fully improves the compression performance of the proposed autoencodr.Then,considering that each pixel in the panorama image has position-independent weight in the projection,the weighted loss function is proposed.In the function,the weight term is specifically designed for each pixel,which ensures that the autoencoder can match the projection process of panoramic image.Finally,combined with the symmetrical structure of the designed compression autoencoder,the greedy block-wise training method is designed.The proposed training method accelerates the training speed of the compression autoencodr,avoids gradient vanishing problem,and improves the overall performance of the network.Next,this paper studies the efficient coding of stereo videos in object panorama applications.While coding and transmission,object panorama applications should not only guarantee the quality of stereo videos,but also the quality of rendered virtual view videos at user side.Therefore,starting from the structural similarity index measurement(SSIM)quality of virtual view video,the paper derives the mathematical relationship between stereo video distortion and virtual view video distortion,and constructs a complete virtual view video distortion model.Then,the virtual video distortion model guides the bit allocation between the texture video component and depth video component in stereo video to improve the coding performance.Next,based on the bit allocated depth videos,the paper designs a method to measure the quality of depth videos by virtual viewpoint quality via the inverse mapping relationship in view synthesis.Based on the distortion of the depth video block,an adaptive threshold is determined to classify the depth video blocks into two kinds.Then,the rate-distortion(R-D)model and rate-quantization(R-Q)model of the depth video blocks are established.Based on the models,the block-level bit-rate allocation is realized.Furthermore,using the established model,the paper also designs a rate distortion optimization(RDO)algorithm for the stereo videos,which further improves the coding efficiency of stereo videos in object panorama applications.Then,the paper examines the actual working requirements of various capture devices in panorama applications.Considering the limitations of the capture devices in computing resources and energy,a low-complexity,low-energy encoder is designed to meet the actual demands of these devices.In detail,based on the theory of compressed sensing,this paper designs a complete codec for image and video in panorama applications.The designed encoder can compress panorama images and stereo videos with low-complexity and lowenergy consumption,which can fully meet the actual needs of the capture devices.At the same time,the paper specially designed an adaptive wavelet decomposition algorithm based on local entropy minimization and a motion estimation algorithm based on block average.The former one can separate the high and low frequency components of the image or video block while reducing the amount of local information,which also lays a foundation for the quantization and sampling.The latter one balances complexity and performance,and enables fast motion estimation,which reduces temporal redundancy between successive stereo video frames.Besides the designed video encoder,this paper also explores the influence of quantization parameter(QP)and measurement rate on coding rate and coding distortion.Based on this relationship,a joint optimization algorithm of QP and measurement rate is established to realize the bit-rate allocation at block level.Thus,the limited bits can be reasonably allocated according to the characteristics of different blocks,which further improves the overall coding performance of the proposed encoder in panorama applications.Finally,this paper summarizes the above research contents and innovation points,then forecasts the development trend and future research direction of panorama technology.As a research on image and video compression for panorama applications,this paper provides new research ideas,solutions and development directions for the compression of panorama images and stereo videos in panorama applications.