Study On Three-Dimensional Video Coding

Three-dimensional video enables viewers to freely choose an arbitrary view-point and viewing direction, and provides three-dimensional visual perception to viewers. It can find wide applications in three-dimensional television, entertainments, video phone, video surveillance, exhibition, education, medical care and military field. Typical three-dimensional video data is comprised of multi-view video and corresponding depth image sequences. The huge amount of information in three-dimensional video is one of the key enabling factors for its wide applications. Therefore, kinds of three-dimensional video compression techniques have been intensively studied in recent years. Especially, the standardization of H.264/AVC based three-dimensional video coding scheme has recently become one of the main activities of moving picture experts group (MPEG).This dissertation investigates H.264/AVC based three-dimensional video compression algorithms and related techniques. Major contributions of this dissertation are summarized as follows:1. A depth based image region partitioning method is proposed for multi-view video, with which the disparity of each image region can be estimated simultaneously. Existing depth based region partitioning algorithms share one characteristic: pixel-wise or block-wise depth disparity field needs to be estimated firstly, and then region partitioning is performed by classifying these pixels or blocks into different groups. Distinguished from these algorithms, the proposed algorithm can directly get an estimation of the disparity for each of the regions with different depth characteristics. Then region partitioning is performed by specifying an optimal disparity from the estimated regional disparities for each block in the image.2. Existing predictive coding methods for motion information in ordinary two dimensional video coding and scalable video coding schemes are summarized and analyzed firstly. Then an inter-view motion predictive coding method, i.e., fine-granular motion matching based motion skipped coding mode is proposed for multi-view video coding. Motion skip mode is an existing inter-view motion predictive coding method, with which the bits for coding motion information of a macroblock can be saved, hence the compression efficiency of multi-view video coding can be improved. The proposed fine-granular motion matching algorithm searches the encoded neighboring views for the motion that matches the motion of the coding macroblock best, and then uses the best matching motion information in the existing motion skip mode. Therefore, the coding efficiency of the existing motion skip mode can be significantly improved. The proposed technique had been adopted into the reference software of multi-view video coding by joint video team (JVT).3. There are strong similarities between video pictures and corresponding depth images in the aspects of contour and motion of video objects. To exploit this kind of redundancy, a joint video-depth coding scheme is proposed to reuse the motion information of encoded video pictures in the coding of corresponding depth images by two motion reusing mechanisms, i.e., motion information copy and motion information prediction. In addition, we also made a preliminary investigation on the prediction structure of joint multi-view video-depth coding, and proposed a prediction structure that can incorporate various existing coding tools that can be used to remove all kinds of redundancies in multi-view video and depth data.4. Video pre-processing prior to video coding can be used to remove or reduce various noises and distortions introduced in the video capturing process, and can enhance the efficiency of subsequent video coding. Automatic exposure control (AEC), one of the most important video pre-processing techniques, is studied in the dissertation, and a luminance histogram based AEC scheme is proposed. The proposed algorithm finds out regions-of-no-interests (RONI) in a captured video picture based on the luminance histogram distribution, and puts the emphasis of exposure on regions-of-interests (ROI) by assigning a relatively small weighting factor for ROI when calculating luminance average. Therefore, the exposure of captured video pictures is optimized.