Reseach On Depth Image Based Rendering For3DTV/FTV

In recent years, information processing, transmission and integrated circuit technologies arerapidly developed. Multi-view video through a variety of media, such as hard disk memory, cable,satellite transmission, terrestrial broadcasting, computer networks, and other various channels goesinto millions of households. And more and more consumers can enjoy the high quality of visualfeast with immersive sense of immersion and realism. Also, multiview video inestimableapplication prospects in related field, such as military, aerospace, detection, and sporting events.The virtual view rendering is one of the key technologies to achieve3DTV/FTV system. Withgeometry information of scene, multi-view color plus depth video (MVD) represents effectiveformat of3DTV/FTV system. MVD technology not only requires a lot of storage and transmissioncapacity of multimedia devices, but also needs to address the problem of multi-view video efficientcompression. Researchers have explored MVD video encoding method, which mainly focuses onthe redundancy between temporal, spatial of the adjacent pixels and the different views. Nowadays,MVD coding technology more considers depth image based rendering technology in the client togenerate a more view video information for the users viewing stereoscopic video at any position,and also to some extent reduces the amount of data in the process of video encoding, transfering. Inthis paper, to solve the problems still existing in the virtual view based on the depth image basedrendering technology, the correlations in video sequences, such as the temporal, spatial andinter-view, are carried out to the research.(1) Virtual view rendering is one of the key technologies to realize three-dimensionaltelevision (3DTV). Because of the depth value’s discontinuity in the depth map, it generates lots ofholes in the virtual view image. For the problem, this paper proposes depth map pre-processingmethods at first, then renders the virtual view images by3D image warping, counts the number ofholes and evaluates the quality of virtual view image to evaluate the results of the ways ofpre-processing. Experimental results demnostrate that it can effective reduce the number of holesand improve the quality of virtual view image by pre-processing.(2) This paper presents a novel fast method of virtual view rendering that the holes are dividedinto large and small hole regions, which can be processed according to their size, also we proposeusing a new way of rendering that some pixels refer to the warping relation of previous frame andthe other pixels update the warping relation by re-computing in the current rendering frame, in theprocess of selecting threshold to determine the two parts of pixel regions, we propose amathematical model to adaptive select the threshold after a large number of experiments.Experimental results show that the proposed method is effective, and it effectively reduces the timeof rendering within a certain range of the quality of virtual image declining. (3) Depth Image Based Rendering (DIBR) is one of key technologies to realize virtual viewgeneration for three-dimensional television (3DTV) and free-viewpoint video (FVV), which canprovide high quality and immersive experience to end viewers. However, artifacts of the renderedimages, including holes caused by occlusion/disclosure and boundary artifacts, may degrade thesubjective and objective image quality. To handle these problems and improve the rendered imagequality, we present a novel view-spatial-temporal post-refinement method for virtual view imagerendering, in which new holes filling and boundary artifacts removal technologies are proposed.The pixels of large holes and boundary artifacts of the virtual view images are filled or updatedwith the spatial-temporal and inter-view correlated reference frame. Also we propose a scheme ofoptimization on optimal reference frame selection to have a better trade-off between thecomplexity and image quality. Experimental results show that the proposed method is effectiveto tackle holes and boundary artifacts problems in DIBR. It can achieve average PeakSignal-to-Noise Ratio (PSNR) gain from0.1dB to1.0dB and Structure Similarity (SSIM) gainfrom0.002to0.008for the test multiview video sequences compared with the state-of-the-artDIBR post-processing schemes.(4) Depth image based rendering (DIBR) is a key technology for achieving three-dimensionaltelevision (3DTV). However, virtual view image synthesized by DIBR is very sensitive to thedepth distortion in depth coding. The depth distortion will lead to geometrical rendering positionerror, and seriously affects the quality of synthesized images. In this paper, we propose a linearweighted filter to minimize the rendering distortion at cost of transferring extra filter parameters assupplement information. A good trade off between extra bit rate for filter parameters and renderingdistortion has been achieved by considering the spatial-temporal correlation of the3D videosequence. The simulation results reveals that the proposed method can significantly improve therate distortion performance, which increases0.41dB to1.08dB Bjontegaard Delta PeakSignal-to-Noise Ratio (BDPSNR) gain compared with the benchmarks.In this paper, for the problems of virtual view rendering, such as boundary artifacts,holes and complexity, we propose the new corresponding soluations. Compired withbenchmarks, the proposed methods achieve great virtual image quality and low complexity.For the distortion of virtual view image from depth distortion, we propose a new method ofrendering distortion elimination, and achieve better Rate-Distortion (RD) performationcompared with the state-of-the-art scheme. And these issues are effectively dealed with, acertain foundation will be laid for the application of3DTV/FTV.