Research On Virtual Viewpoint Rendering Algorithm Based On Depth Image

With the high-speed development of the communication system,there has been a great transformation from the traditional 2D media toward three-dimensional(3D)media recent years.It is used in various applications widely such as 3DTV,3D cinema,home theater,PC games,sports,education,medicine,business,computer visualization and so on,it provides people with a more visual experience.Free-viewpoint television(FTV)has attracted much attention because it can realize the construction of 3D without the help of any auxiliary tool.Depth-image Based Rendering(DIBR)is the key technology in FTV system.DIBR can draw a virtual image through a texture map and its corresponding depth map.Compared with model-based rendering and image-based rendering,DIBR has the advantages of small bandwidth requirements,the small number of input images and faster rendering speed.DIBR has the high speed of warping,Double-sided DIBR can achieve complementary of occlusion region information using the left and right views.DIBR is used in 3D-HEVC coding technology,Double-sided DIBR is used in VSRS software.However,whether DIBR or Double-sided DIBR,they still have the problems of holes,resampling,overlapping and edge ghosting.Double-sided DIBR also has high computational complexity.Those problems have affected the quality of the synthetic viewpoint.In this paper,the problems of holes,resampling,overlapping in DIBR and the problem of high computational complexity in Double-sided DIBR are improved.This paper analyzes the causes and solutions of holes,resampling,overlapping and edge ghosting in DIBR,the main work of this thesis is as follows:(1)Aiming at the problem of holes generated in the Depth-image Based Rendering technology,an improved Criminisi algorithm based on the combination of depth and gradient was proposed.The priority was improved firstly,an exponential confidence term and data term were added to enhance the details filling.When searching for a matching block area,a new color matching factor was adopted,a gradient factor was added,the texture image and depth image were matched combining with depth factor after warping.It fills the hole in the image effectively.(2)Aiming at the problem of resampling and overlapping generated in the Double-sided Depth-image Based Rendering technology,an improved OPFD algorithm based on a method of judging integer pixel was proposed.A new method of determining the sub pixels was used firstly.Secondly the method of judging the hole of integer pixels was used,the required integer pixel depth values were inferred by using the surrounding sub pixel information.Thirdly eligible sub pixels were chosen by defining appropriate initial depth thresholds and limits depth thresholds.Lastly sub pixel fusion formula was used so that making the depth values of sub pixels and integer pixels consistent and avoiding the error of pixel fusion.Experimental results indicate that the improved method has the better subjective and objective evaluation compared with traditional hole filling algorithm,and the time complexity is reduced,the operating time is shorten.(3)Aiming at the problem of holes,resampling and overlapping generated in the Doublesided Depth-image Based Rendering technology,an improved Forward and Reverse Wrapping DIBR technology was proposed.The technology improves the Double-sided DIBR technology from four aspects mainly.A depth difference estimation algorithm was proposed firstly,the accuracy of finding the main reference image is raised.Secondly,the improved OPFD algorithm based on Z-buffer was used in 3D-warping,the problem of resampling and overlapping was resolved effectively.The improved Background-based holes filling algorithm was used for depth virtual image to eliminate holes thirdly.Forthly the Forward and Reverse Wrapping process was improved,through judging the consistency of background of blocked information of projected image and auxiliary color reference image,the different holes filling algorithms were selected to fill the holes in the color virtual image so that to achieve a better holes filling effect.The improved technology reduces the complexity of the algorithm significantly.Experimental results indicate that the improved method enhances the quality of picture,PSNR and SSIM compared with traditional technology.