Research On The Key Technologies Of Editing In Stereoscopic Image And Video

With the rapid development of 3D hardware and software technologies in recent years, a number of 3D resources are widely applied in various areas, such as entertainment, industry, education, military medical treatment, etc. Mean-while, people can also easily view 3D content from the theater, TV, PC, mobile devices and many other display medias. However, currently 3D resources of high quality are still very scarce, which hinders further popularization and ap-plication of 3D technology. Due to the complex characters of human visual sys-tem, many factors should be considered in 3D content acquisition and editing in order to produce the visual experience similar to the real world. Moreover,since the binocular stereoscopic images and videos contain the depth informa-tion from different views, existing 2D content editing and processing technol-ogy can not be directly applied to the 3D content. Based on these observations,in this paper we analyze the binocular stereo imaging and human stereo percep-tion, focus on the issue of stereo content editing for stereo effect enhancing, and conduct our research from three aspects including stereoscopic camera control,stereoscopic image depth adjusting and stereoscopic image retargeting. The main contributions are as follows:(1) Aiming to ensure the visual quality in stereoscopic video editing and production, we presents an automatic stereo camera control method based on assessment of 3D video quality of experience(QoE). We introduce an at-tention tracking mechanism to initialize the convergence plane so that the accommodation-vergence conflict and stereo window violation can be avoided as much as possible, and provide novel functions to smooth the change in con-vergence planes in order to avoid depth jump over time. Since the acquired parameter setting may also not create high 3D QoE, we we also present a direct and objective method for 3D video QoE assessment to quantify the result of stereo parameter adjustment in editing and production workflow. The results of the 3D video QoE assessment are used as the feedback for optimizing the obtained stereo camera parameters.(2) For depth adjustment of stereoscopic images, we propose a nonlinear disparity remapping method based on image patch matching and stereo com-fort metric. Our method can adjust the depth range of stereoscopic images as well as selectively adapt depth of the local visual important regions. We present a visual comfort measurement model to quantify the visual experience of stereoscopic images, which can provide an accurate and reliable basis for the disparity adjustment process. In addition, we present a patch-based stereo image processing method to achieve the depth adjustment in order to minimize the shape distortion of the image scene. The experimental results show that the proposed method can improve the visual comfort and perceived depth of the stereo image, and preserve the 3D structure and depth continuity.(3) Aiming at the adjustment of stereoscopic images on various display devices , we propose a depth carving method for stereoscopic image retargeting according to scene depth structure. We use the disparity correspondence and the image segmentation to model an image scene as a 3-D space. Given the depth order of the objects in the image scene, we can remove not only the seams in the background regions but also the seams along the boundaries of objects.As a result, the shape of the objects in the image scene can be preserved and the background are not deformed obviously. The experimental results illustrate that our method can minimize the object distortion and preserve the disparity consistency of the retargeted images.(4) Design and implementation of stereo video and image editing system.To evaluate the effectiveness of the proposed system, we design a stereoscopic content editing system. The experiment results show the proposed methods can help stereographers produce and edit stereoscopic video more quickly and accurately. Moreover, the issues of stereoscopic image depth adjustment and display adaptation can be handled efficiently. By using the proposed system,the stereo visual experience can be enhanced obviously on various display de-vices.