Interactive Streaming For Immersive Video

Nowadays,the rapid development of network technology repeatedly refreshes peo-ple's traditional understanding for perceptual experience.To promote both reality and interaction will be the main development tendency of next generation digital media.In this process,the immersive video has become the most typical sort of media by provid-ing the immersive experience.Among a variety of immersive videos,multiview video and 360° video have highly flexible interactive capabilities,which have attracted exten-sive attention due to their support for users to be "viewpoint" drivers.Nevertheless,the improvement of interactive immersive experience is at the cost of transmitting huge multi-dimensional video data,hindering the development of online real-time services.Based on the carrying capacity of current network,this thesis aims to maximize the visual qual-ity and smooth navigation experienced by the user,and engages in three key problems in multiview video and 360° video streaming,i.e.,robust transmission,real-time interaction and efficient compression.The main contributions are the following:(1)Aiming at the robust problem in interactive multiview video streaming,we propose a joint packetization strategy for texture image and the corresponding depth map.Various packet loss events and bitrate levels are considered.The performance and real differences between two packetization strategies are analyzed in theory.Exper-imental results show that,compared with the independent packetization approach,gains up to 2.32 dB and 1.55 dB are achieves by the proposed scheme for single views and virtual viewpoints,respectively,and the robustness in multiview video streaming is improved.(2)Aiming at the real-time problem in interactive multiview video streaming,we pro-vide a problem formulation to simultaneously optimize the subset of camera views and their encoded video bitrates for downloading,where the navigation window for the user is introduced.An optimal algorithm based on the dynamic programming and a suboptimal greedy algorithm with low complexity are developed to solve this optimization.Experimental results show that,compared with alternative adaptive streaming algorithms,both proposed solutions achieve significant improvement in terms of navigation quality and enhance real-time performance in multiview video streaming.(3)Aiming at the compression and real-time problem in interactive 360° video stream-ing,we provide a problem formulation to simultaneously optimize tiling size,dis-tortion version of each tile and the head-angle-to-stream mapping function.An alternating and iterative solution is developed to solve this optimization.A sparse directed graph is built to learn the head movement prediction,where a saliency map,collected viewers' head movement traces,and a human biological head rota-tion model are aggregated into a 1-hop Markov model.Experimental results show that our head movement prediction scheme outperforms existing proposals,and the optimized video streaming improves by up to 3.9dB in expected PSNR compared with the non-optimized one.Both compression and real-time performance in 360°video streaming is enhanced.