Stereo Image And Video Editing

With the success of the3D movie "Avatar", stereo videos have become very pop-ular in recent years. In general, each stereo image contains two regular2D images captured from the same scene at the same time but from slightly different viewing loca-tions. When a stereo image/video is displayed on the screen, with appropriate devices, viewers see one2D regular image/frame with the left eye and the other with the right eye. The human brain will then fuse the two images/frames together to produce3D scene depth information. As stereo images can convey more visual information, stereo media are considered as one of the main research directions of future development.Although there are a lot of tools available for editing traditional2D imags/videos, tools for editing3D media are very limited. In general, editing and processing stereo images/videos are more difficult than those of2D images/videos, due to three major reasons. First, it is difficult to obtain noise-free and accurate disparity/depth maps for stereo images/videos. Stereo matching methods, which aim at finding correspondences between pixels in the left and right images, generally do not perform very well, espe-cially for stereo images of natural scenes. Even we use a depth camera, to obtain high resolution and noise-free depth maps from the low resolution and noisy output is still difficult. Second, it is difficult to ensure the spacial coherence between left and right images of stereo image pair, which is very important for minimizing distortion and producing high quality results. In practice, the left and right images usually need to be simultaneously processed in order to enforce the coherence between left and right images, such as processing by a global optimization. Thus, algorithms for processing stereo media are usually more complex than those for2D media, with high computa-tional and memory costs. Third, we need to ensure both motion and depth coherences across neighboring frames. In this thesis, our aim is to discuss fundamental problems existing in stereo image and video editing, at the same time attempts to address these technical difficulties and provides users with a number of editing methods for process-ing stereo images/videos. We mainly introduce three editing methods as follows. First, we propose a novel depth mapping method for stereo video depth mapping. Most stereo videos are developed primarily for viewing on large screens located at some distance away from the viewer. If we watch these videos on a small screen lo-cated near to us, the depth range of the videos will be seriously reduced, which can significantly degrade their3D effects. In order to address this problem, we propose a linear depth mapping method to adjust the depth range of a stereo video accord-ing to the viewing configuration, including pixel density and distance to the screen. We also consider characters of human binocular vision, such as relative depth among objects to depth perception, human eyes sensitivity to straight lines and planes. Our method tries to minimize the distortion of stereo image contents, by preserving the relationship of neighboring features and preventing line and plane bending. It also considers motion and depth coherences across neighboring frames. While depth co-herence ensures smooth changes of the depth field across frames, motion coherence ensures smooth content changes across frames. Our experimental results show that the proposed method can improve the stereoscopic effects while maintaining the quality of the output videos.Second, in order to obtain high quality depth mapping and other stereo editing effect, we extend the shift-map method for stereo image editing. Our method simulta-neously processes the left and right images on pixel level using a global optimization algorithm. It enforces photo consistence between the two images and preserves3D scene structures. It also addresses the occlusion and disocclusion problems, which may enable many stereo image editing functions, such as depth mapping, object depth adjustment and non-homogeneous image resizing. Our experimental results show that the proposed method produces high quality results with a number of editing functions.Third, we propose a method for creating infinite stereo panoramas. A stereo in-finite panorama is a panoramic image that can be infinitely extended by continuously stitching together stereo images that depict similar scenes, but may be taken from dif-ferent geographic locations. It can be used to create interesting walkthrough environ-ment. An important issue underlying this application is to seamlessly stitch two stereo images together. Although many methods have been proposed for stitching2D images, they may not work well on stereo images, due to the difficulty in ensuring disparity consistency. In this thesis, we propose a novel method to stitch two stereo images seamlessly. We first apply the graph cut algorithm to compute a seam for stitching, with a novel disparity-aware energy function to both ensure disparity continuity and suppress visual artifacts around the seam. We then apply a modified warping-based disparity scaling algorithm to suppress the seam in the depth domain. Our experimen-tal results show that the proposed stitching method is capable of producing high quality stereo infinite panoramas.