| At the convergence of computer vision, graphics, and multimedia, the e-merging 3D video technology promises immersive experiences in a truly seamless environment. Different from traditional 3D graphics,3D video is captured and reconstructed from the real world in real time while the 3D attribute allows the video objects to be rendered at arbitrary viewpoint in terms of the image based rendering technologies.As the important assistant information, depth data is widely used in image based rendering schemes. Thus, many depth sensors are developed to capture the depth information with lower cost and higher precision. As a typical depth sensor, Kinect is widely used in many related 3D video applications. However, due to the cost constraint, the quality of Kinect depth is inadequate. In this thesis, we first propose a texture assisted Kinect depth inpainting scheme to improve the Kinect depth quality in which the texture edge information is extracted to partition the depth map as different regions and distinct diffusion-based inpainting algorithms are designed to fill the hole and align the boundary between texture and depth. Comparing with the original depth, the inpainted depth map enhances the quality of advanced processing.In many scenarios, the depth data should be transmitted to remote side for the further processing. Efficient depth compression scheme is essential for practical depth transmission system. In this thesis, we propose a Kinect depth compression scheme to exploit the correlation in spatial and temporal domains while preserving the depth features for further applications. The Kinect depth is first reformed by divisive normalized bilateral filter (DNBL) to suppress the depth noises and recover the correlation in spatial domain. Before the traditional video coding, the inter-frame correlation is exploited by proposed 2D+T prediction, in which depth volume is developed to simulate 3D volume as the prediction reference for active region detection. The active region is fed into the video encoder for traditional intra and inter prediction with residual coding, while the inactive region is skipped during depth coding and reconstructed by 3D reference surface in decoder.The high dynamic range (HDR) depth data is widely used with higher pre-cision than the traditional 8-bit depth data. In this thesis, we propose a layered HDR. depth compression framework based on 8-bit image/video encoder to achieve efficient compression with low complexity. With respect to the characteristics of HDR depth, a depth map is partitioned into two layers:the most significant bit (MSB) layer with rough depth value distribution and the least significant bit (LS-B) layer with details of depth value variation. An error-controllable pixel domain coding scheme is proposed in MSB layer to efficiently compress the general depth information with sharp edge while the transform based coding scheme is employed in LSB layer. The data format in LSB layer is guaranteed within 8-bit data so that the existing 8-bit image/video codecs can be fully utilized. The proposed coding scheme can achieve real-time HDR depth compression with satisfactory reconstruction quality.As a popular 3D video format, free viewpoint video (FVV) services are draw-ing great attention. People desire to watch FVV not only on PC but also on mobile devices. However, FVV on mobile devices over cellular network is very challenging due to the requirement for large bandwidth and limitation in compu-tation on mobile devices. To address such challenges, in this thesis we propose a cloud-based FVV rendering framework for mobile devices. In this framework, cloud performs rendering for mobile devices. In order to improve the user ex-perience, a novel resource allocation scheme is proposed, which jointly considers rendering allocation between cloud and client and rate allocation based on rate-distortion analysis, to reduce the interaction delay and improve the visual quality on mobile devices.
|
PDF Full Text Request