Research On View Synthesis Based On Depth-image-based Rendering

3D TV has made high speed of development and application in recent years due toits exceptional visual experience enhancement. The key technology of3D TV is viewsynthesis based on depth-image-based rendering (DIBR), which can synthesizearbitrary novel views conveniently and edit3D content in real-time. It provides a3DTV system with stronger interactivity. We call this kind of TV editable3D TV.The purpose of this dissertation is to remove folds and holes in synthesized viewsby researching on view synthesis based on DIBR to improve image quality, which willprovide the theoretical and technical support for the application of DIBR in3D TVsystem. This dissertation aims at obtaining high quality of synthesized views, andimproving the applicability of the algorithm for hardware implementation. Anotherpurpose of this dissertation is to pursue good3D image quality when the baseline islong.In this dissertation, the theoretical basis of view synthesis based on DIBR is firstlyintroduced. The parallel camera setup model, which is currently used in3D TV, ismainly discussed. Then based on this model, two key technologies are studied: foldselimination and hole-filling. Finally, the dissertation presents the hardwareimplementation of folds elimination approach and hole-filling algorithm.The main research results of this dissertation are summarized as follows.①With respect to folds, which may appear when the DIBR technique is used togenerate destination image, occlusion-compatible algorithm is investigated. Based onthis algorithm, an approach to eliminate folds based on view judgment is proposed. Thisapproach can determine the scan order of reference image by whether destination imageis left view or right view, so as to remove folds. In addition, this approach does not relyon calibration parameters of camera and content of image. Experimental results showthat the proposed approach can eliminate folds with lower complexity in contrast to theZ-buffer algorithm, and it is adapt to virtual view rendering when the virtual viewpointshifts horizontally.②To be convenient for hardware implementation as well as achieve good imagequality, a hole-filling algorithm based on disparity map is proposed. One importantaspect of the algorithm is that the disparity map of destination image is used for holefilling. Firstly, holes detection based on disparity map is conducted, and the start point and the end point of the hole are recorded. Then foreground pixels and backgroundpixels are distinguished for hole dilation according to disparity map, so that areas withmatching errors can be determined and eliminated. In addition, parallaxes of pixels inthe area with holes and matching errors are changed to new values. Finally, holes arefilled with background pixels from reference image according to these new parallaxes.Experimental results show that the quality of the novel view after hole-filling is quitewell; and geometric distortions are avoided in destination image, in contrast to thevirtual view generated by depth-smoothing methods. Moreover, this algorithm is easyfor hardware implementation.③As for the case of long baseline, characteristics of holes which may be found inthe novel view synthesized by DIBR are discussed. A new hole-filling algorithm basedon inpaniting is given. This algorithm combines with the hole-filling alogrithm based ondisparity map, so it can fill big holes with good image quality as well as accelerate thespeed of filling.④With low hardware resoureces, the approach to eliminate folds based on viewjudgment and the hole-filling algorithm based on disparity map are implemented on adevelopment platform of FPGA. The hardware architecture, timing diagram andsimulation results for hole-filing algorithm are analyzed.As an original work, the author tried her best to make a break through on researchmethods and research thoughts, and the research results will provide new methods forthe practical applications of view synthesis based on DIBR in3D TV system.