Research On Semi-automatic2D-3D Conversion Algorithm

With the growing concern of3D TVs and3D videos, the3D industry is developing rapidly. But now, the shortage of3D videos is an important factor that hinders the development of3D industry.A3D video is made up of two slightly dissimilar videos, called left-and right-eye video. These two videos are displayed onto the screen through stereoscopic display technology to ensure that the left and right eyes of the viewer view the corresponding videos simultaneously. Then the viewer can automatically feel the sense of the depth, thereby forming a stereoscopic feeling. There are a variety of methods to produce3D video, mainly classified as three categories:direct acquisition of the left and right eye video through dual cameras; using a camera plus a depth camera to acquire a color video and a depth video; using a camera to capture a color video and then applying video processing algorithms to estimate the corresponding depth of the video. The last one is called the2D-3D video conversion.Among these methods,2D-3D conversion only requires an ordinary camera. In addition, the vast existing2D videos can be converted to3D by this method but other methods fail to do this. Therefore,2D-3D conversion is the cheapest and most practical3D production method. According to the extent of human involvement, the existing2D-3D conversion system is divided into the automatic and semi-automatic system.In the semi-automatic system, two improved methods are proposed to tackle the problems in key-frame selection and depth propagation. First, a novel key-frame selection algorithm is proposed based on accumulated occlusion. It combines the frame interval and the motion information and uses the cumulative occlusion to measure the depth propagation errors. Then based on this cumulative occlusion curve, the keyframe is selected in the points with big error. The experimental results show that, compared with the previous program which selected key frames according to a fixed frame interval, the proposed algorithm can keep depth errors of all frames in the whole video controlled in a lower level and thus improve the overall3D effect. Second, inspired by the color consistency in the coloring theory, a depth propagation algorithm based on depth consistency is proposed. Depth consistency refers to that two neighboring pixels in space should have similar depth values if their intensities are similar. Based on this principle, depth propagation in2D-3D is modeled as a constrained optimization problem. Experimental results show that compared with the shifted bilateral filtering (SBF) algorithm, the proposed algorithm not only holds a similar performance, but also increases the processing speed by nearly5times on average. Third, in terms of the application of the algorithm in the semi-automatic image2D-3D, an interactive software platform is developed. Through this software, users simply scribble on the input image and set the depth values. The high-quality depth map can be obtained. This software is simple, convenient and practical.In terms of the automatic system, two kinds of2D-3D conversion softwares are developed for single image and video conversion, respectively. The image2D-3D conversion software provides two depth estimation algorithms. After an image is inputed, its depth map is estimated accmatically and then3D picture is generated. In the video2D-3D software, the depth estimation combines color and motion depth cues. It automatically performs video depth estimation and video stereoscopic synthesis. These two softwares can both reach the similar performance of the2D-3D conversion in the existing TV products. They can be widely applied in3D TV and3D chips.