Research On Free-viewpoint Video Virtual View Rendering Technique

Free-viewpoint video is based on 3D video,allowing people to view 3D scenes by freely changing the viewpoint.In the Free-viewpoint video research,virtual view synthesis technology occupies an important position.Depth-Image-Based-Rendering(DIBR)techniques have the advantages of fast speed,saving bandwidth and good rendering effect,which have aroused wide attention from scholars both at home and abroad.However,when rendering a target point of view with these techniques,these images often exit holes problem,overlap problem and resampling problem and many other issues.In order to solve these problems and improve the quality of virtual view rendering,scholars have put forward many excellent algorithms.Based on the previous research,this paper presents a dual viewpoint reference rendering virtual viewpoint method and two kinds of single viewpoint reference rendering virtual viewpoint method for different application scenarios.The main research contents of this paper are as follows:(1)This paper presents a virtual view rendering method based on deep image segmentation.This method uses two reference viewpoints for virtual view image rendering.Firstly,the Canny operator is used to detect the depth map,and the untrusted region is marked according to the direction of rendering.Then,the depth image is processed by multi-threshold image segmentation.And this image is used to guide the three-dimensional image transformation,during which supplemented by local mean filtering.Finally,the obtained virtual view image subjective quality is good,and the PSNR of virtual view could be 0.74 dB higher than that MPEG-3DV/FTV reference software.(2)In order to obtain a high quality virtual view image with limited bandwidth,a virtual view rendering method based on depth guided void filling is proposed.This method uses only a single reference image,so it can save more than half the bandwidth compared to multi-reference rendering.This method is mainly composed of deep interpolation algorithm based on depth interpolation algorithm and region-based holes filling.The former is mainly used to solve the problem of small holes in the rendering process,while the latter is to solve the large holes problem in rendering process.Based on Criminisi algorithm,this method improves the calculation of the priority and the selection of the best matching block,and selects the best search area according to the different regions.Finally,the subjective and objective quality of the virtual view image is better than the other contrasted algorithm.(3)Aiming at the background of complex scenes,this paper proposes a virtual view rendering method based on the combination of spatial and temporal.This method mainly uses the Gaussian Mixture algorithm in the machine learning to implement the background modeling,train and extract the background of the video sequence,and then use the K-means algorithm to divide the depth map into foreground and background,according to the time domain correlation to get the depth information.And the extracted information after three-dimensional image transformation is applied to the holes filling,and the filling method can be improved for different motion characteristics.The experimental results show that the virtual view image obtained by this method can meet the requirements of human visual system.