Research On 3DOF+ Video Transmission Technology Based On Cloud Rendering

In recent years,with the continuous development of virtual reality technology and the improvement of the availability of related equipment,the immersive media has attracted more and more attention.In addition to providing full-angle vision in three rotation dimensions,3DOF+ video also allows users to perform small-scale movements of the viewpoint in three directions: forward and backward,left and right,and up and down.In 3DOF+,the rendering system needs to use panoramic video information with limited viewpoints to generate video content for the user's current viewpoint position.At the same time,processing and transmitting high-resolution,high-bit-rate panoramic videos require a large amount of computing power and network bandwidth resources.This paper focuses on 3DOF+ rendering quality and actual transmission applications.A panoramic virtual viewpoint synthesis algorithm based on improved triangle interpolation is proposed,which improves the quality of the synthesized virtual view.Aiming at problems such as insufficient computing power of mobile terminal devices,a practical low bitrate,lightweight terminal and cloud-Rendering based 3DOF+ video transmission system is designed and built.Aiming at the problem of how to draw the virtual view of panoramic video with high quality,this paper proposes a panoramic virtual viewpoint synthesis algorithm based on improved triangle interpolation.First,the reference view is pre-processed and edge-divided,and the edge and non-edge parts are marked.Then,in the method of triangle mapping,for nonedge part,weighting is used for interpolation calculations taking into account factors such as the depth value of pixel point,the shape of the mapped triangle,and the position of the point to be interpolated within the triangle.For the edge part,the nearest neighbor interpolation method based on Manhattan distance is used.Finally,a virtual view of the user's new viewpoint is synthesized through the multi-view fusion and hole filling algorithm.The experimental result show that the subjective and objective quality of the solution is better than the current mainstream panoramic video virtual viewpoint synthesis algorithm,and the average PSNR is improved by 0.156 d B.A good 3DOF+ user experience mostly depends on high-performance devices for local computing and rendering.At present,the device performance of mobile terminals often does not have such rich computing power resources.Aiming at such practical problem,this paperdesigns and sets up a 3DOF+ video transmission system based on cloud rendering.The system mainly utilizes the powerful GPU computing resources in the cloud to improve the system's logical computing,image processing and rendering capabilities.On the server side,the entire workflow of 3DOF+ video generation,rendering and compression encoding is implemented.On the client side,all need to do is decoding and rendering the video.This solution breaks through the limitation of terminal performance,and makes up for the shortcomings of insufficient computing power of user terminals.At the same time,experimental result show that the system reduces bandwidth requirements by up to 87.5% compared to the scheme of transmitting multiple complete panoramic textures and depth maps.The system can provide users with the corresponding 3DOF+ video of the viewpoint,which significantly improves the user's strong sense of interaction and enhances the user's sense of immersion and presence.