Research On Key Technologies Of2D-to-3D Conversionbasedon Depth-image-based Rendering

With the success of “Avatar”,3D TV steps into our lives. People desire a wonderful3D experience through TV at home, which gives an impetus drive to a number of highquality3D contents to be produced. However,3D content generation is atime-consuming and high cost process at present, which causes the lack of3D contents.Fortunately, there are vast2D videos, we can convert this2D videos to3D contents toalleviate this problem. One of the key technologies of2D-to-3D conversion is depthestimation. The accuracy of depth map estimated by depth estimation algorithms has agreat influence on the performance of synthetic3D image. In addition, visual comfortimprovement is also one of the key technologies of2D-to-3D conversion, which canreduce visual fatigue while people view3D image.The purpose of this dissertation is to study the existing key technologies of2D-to-3D conversion, and then propose algorithms of depth estimation and visualcomfort improvement that provide better performance, which will provide thetheoretical and technical support for their applications to2D-to-3D conversion.In this dissertation, the theoretical basis of2D-to-3D conversion is firstlyintroduced. Then depth estimation, one of the key technologies in2D-to-3D system, isstudied. The depth estimation algorithms based on definition difference of image andrelative height depth cue are thoroughly studied. In addition, depth estimation algorithmbased on moving object segmentation is also discussed. Finally, visual comfortimprovement based on depth adjustment for depth-image-based rendering (DIBR)system is elaborated.The main research results of this dissertation are summarized as follows.①As foreground is commonly in focus and the objects focused areusually clearerin picture, we can utilize this characteristic to estimate depth. A depth estimationalgorithm based on definition difference of image is proposed. Compared with otherdepth estimation algorithms, the proposed algorithm is of lower complexity and morefacilitative for hardware implementation.②A improved depth estimation algorithm based on relative height depth cue isproposed. The constraint formula is simplified and two new templates, right-to-left andleft-to-right templates, are added for different scenes in the improved algorithm. Inaddition, the filter used in original algorithm is replaced by joint bilateral filter to make the depth map smoother. What’s more, visual attention is introduced so as to improvethe accuracy of depth estimation for Object-of-Interest (OOI). The experimental resultsshow that the proposed algorithm is able to recover relatively accurate depth maps fromoutdoor scenes.③A moving object segmentation algorithm combining symmetric inter-framedifference and joint bilateral filter is proposed, and we apply it to the improved depthestimation algorithm based on relative height depth cue. Firstly, the differences of threeconsecutive frames are used to obtain motion regions (difference map). Then the jointbilateral filtering with a30×30filter, which uses current frame as guide image, isperformed on the difference map to fill holes of the moving object. Next, Canny edgedetection is performed on current gray image and filtered difference image, respectively.The two results are combined to get thecompleted and clearer edges of moving object.Hence a more accurate object can be obtained by applying the edges of moving objectto the post-processing of the filtered difference image. Finally, the obtained movingobjects are applied to the improved depth estimation algorithm based on relative heightdepth cue.④A depth adjustment method for visual comfort improvement fordepth-image-based rendering (DIBR) system is proposed. The3D image warpingequations for DIBR are presented first. As the so-called shift-sensor camera setup isused, stereoscopic images obtained by the3D image warping imply horizontal sensorparallax only. Then, we thoroughly analyze the geometry of3D TV display system, andexpress the perceived depth as a function of horizontal screen parallax. Finally, therelationship between horizontal sensor parallax and horizontal screen parallax is given.Based on this relationship, a depth adjustment method is proposed which controls theamount of horizontal sensor parallax of stereoscopic images by changing the values ofvariables in camera space during the processing of DIBR. Subjective evaluations showthat the proposed depth adjustment method can generate stereoscopic images withdifferent perceived depths that people desire.As an original work, the author tried her best to make a breakthrough on researchmethods and research thoughts, and the research results will provide new methods forthe practical applications of2D-to-3D conversion technology in3D TV.