Research On Virtual View Synthesis Algorithm Of 3DoF+

With the continuous development of virtual reality technology,immersive media has received widespread attention.Among them,the 3 Degree of Freedom Plus(3DoF+)media allows users to freely switch viewpoints within a certain range,and select the panoramic video content they want to watch,which enhance the user's sense of presence.However,due to the limitation of panoramic video collection and transmission,it is impossible to directly obtain video content from any viewpoint.The virtual view synthesis technology can effectively solve the above problems,but the existing technology does not make full use of the reference view information,resulting in the phenomenon that the virtual views are not accurate enough and artifacts appear in the virtual views.It is difficult to generate a highquality virtual view.Therefore,this article focuses on research of virtual view synthesis algorithms of 3DoF+,proposes a multi-view joint virtual view synthesis algorithm and a multi-view-based artifact elimination algorithm,and builds a complete 3DoF+ system based on the above algorithms.The research results of this paper can effectively improve the quality of virtual views and provide users with a better immersive viewing experience.The existing panoramic video virtual view synthesis algorithm performs independent view warping on each reference view,which can not make full use of the information of the reference view.In order to effectively use the information of multi-views,this paper proposes a multi-view joint virtual view synthesis algorithm.The algorithm obtains the subpixel information at the virtual viewpoint by performing three-dimensional space transformation on multiple viewpoints,and analyzes the correlation between these subpixels and the integer pixels to be interpolated.Based on the correlations obtained from the analysis and for different application scenarios,this paper designs and implements three different multi-view synthetic interpolation algorithms,namely nearest neighbor interpolation,triangular interpolation and non-uniform interpolation.Experimental results show that,compared with the existing panoramic video virtual view synthesis algorithm,the nearest neighbor interpolation algorithm has lower complexity;triangular interpolation has more excellent performance in complex texture areas;non-uniform interpolation has a greater improvement in overall PSNR,which can reach 1.062 dB.In scenes with high dynamic depth,artifacts are prone to appear in the virtual view generated by the existing panoramic video virtual view synthesis algorithm,which affects the user's visual experience to a certain extent.In order to solve the above problems,this paper proposes an artifact elimination algorithm based on multi-views.The algorithm first analyzes the spatial characteristics of the artifact area;then combines these features to distinguish the artifact area in the virtual view and the foreground and background scenes in the area.Then,the foreground or background pixels are selected according to the position of the points to be interpolated.Finally,the interpolation is done.Experimental results show that compared with the mainstream virtual view synthesis algorithm of panoramic video,the algorithm proposed in this paper can effectively remove artifacts and improve the quality of the synthesized view.Based on the virtual view synthesis algorithm,this article also builds a complete 3DoF+system.In this system,the server side includes three main modules: main view selection,atlas construction,and compression encoding.The client includes three main modules:decoding,atlas generation,and rendering and display.The server is responsible for preprocessing the input 3DoF+ video to remove redundant description information of the scene,and then send the processed atlas to the encoder for encoding.The client is responsible for synthesizing a virtual view based on the pose information of the helmet and the decoded video,and rendering and displaying it in the helmet.This system provides a good viewing experience for 3DoF+ immersive media users.