Research And Implementation Of Virtual View Synthesis Based On GPU Acceleration

Free viewpoint television(FTV)is a branch of three-dimensional television(3DTV),and it is the direction of future televison.Compared with traditional 3DTV,FTV can provide more viewpoints,bring people a better stereo experience.Virtual view synthesis is the core technology of FTV.Among virtual viewpoint synthesis techniques,the most attention is depth image based rendering(DIBR)techniques.Considering the inaccuracy depth map and changes of the occlusion relations during DIBR,synthetic images often contain ghost effects and some holes of varying sizes.Existing DIBR-based virtual view synthesis algorithms are very time-consuming,and it is difficult to do real-time synthesis of virtual viewpoints.The main contents of this paper are:analyzing the reasons of above problem,and proposing associated solutions.(1)The ghost effect can be seen on the boundary of foreground and background of virtual viewpoint.It mainly caused by the depth discontinuities and color differences of reference views.To cope with the ghost effect,this paper first uses color correction of reference views,and then combines of two fusion methods which are depth-based color image fusion and directly color image fusion.(2)The hole problem is the most important and difficult to solve in virtual viewpoint synthesis.It mainly caused by changes of the occlusion relations during DIBR.To cope with the small holes or cracks,this paper uses depth map pre-processing and inverse warping method.To cope with larger holes or disocclusions,this paper uses the image inpainting with depth map to get a better virtual view.Because the process of image inpainting is very time-consuming,this paper implements the inpainting algorithm on GPU to accelerate the inpainting process based on Compute Unified Device Architecture(CUDA).The experiment results show that our proposed method improved the quality of vir-tual view synthetic images,and the image inpainting algorithm based on CUDA imple-mented also significantly reduce the time of hole filling.