Research On Virtual Viewpoint Synthesis Algorithm Based On Panoramic Video

Nowadays,the virtual reality technology develops rapidly,people are increasingly seeking a more immersive and immersive video experience,and the concept of 3Do F and 3Do F+ has been proposed.3Do F can only provide users with three degrees of freedom based on the rotation of the user's head.3Do F+ is based on 3Do F,which can also provide users with a limited range of forward/backward,leftward/rightward,and upward/downward head movement experience(no walking is allowed).To implement 3Do F+,it is essential to generate a view of any viewpoint through a virtual viewpoint synthesis algorithm by a limited number reference views.In this paper,the virtual view synthesis algorithm based on panoramic video is studied.A triangle-warping algorithm based on spherical relationship is proposed.Compared with the triangular –warping algorithm based on Equirectangular Projection(ERP)plane relationship,the performance of the algorithm is improved.In this paper,for the virtual view synthesis algorithm based on panorama,the problem of interpolating the color value and depth value of the integer pixel point by directly using the reference point distance on the ERP plane at the virtual view point is studied.According to the analysis of the process of the projection from the three-dimensional sphere of the panoramic video to the two-dimensional projection plane of the panoramic video,there is some deformation of distance,angle and area in the process of the projection,which causes a certain irrationality in the triangle interpolation according to the positional relationship of the ERP.On this basis,this paper use the weight of the three reference points that is determined according to the positional relationship of the virtual view in the spherical surface to interpolate the integer pixel.Finally,the performance of the algorithm is measured by the block PSNR.The experimental results show that the performance of the triangular-warping algorithm based on the spherical relationship is better than that of the original ERP-based triangular-warping algorithm in the high latitude of the spherical surface.Based on the virtual view synthesis algorithm of panoramic video,this paper proposes and builds a 3Do F+ system,which uses the head-mounted display device to render video content of any viewpoint position with video information with few viewpoint positions,thus providing users with real and good interactivity and visual experience.The system first uses the VR helmet to track the user's head,then uses the position provided by the helmet to traverse the angle between each camera orientation and the VR helmet orientation,and selects the two cameras with the smallest angle as the reference camera,then synthetic virtual view by the reference view,that in the next step will be rendered and displayed on the VR display device.With the 3Do F+ system,users can perform head movements within a limited range to obtain video experiences from different perspectives.