Study On Coding Of Elemental Images In Integral Imaging

Integral imaging is a technique capable of displaying three-dimensional (3D) images with continuous parallax, full color, continuous viewpoints, and real perspectives, so it has been attracted much attention as a promising 3D display technique. Displaying a high-resolution 3D image requires quite a large number of high-resolution elemental images, which inevitably leads to problems of data transmission and storage. So this paper is concerned with effective and real-time coding of these images. At present, the methods for coding elemental images can be divided into three categories, including the lossless compression methods, the coding methods based on available single-view video compression standards, and the algorithms proposed especially for elemental images taking their special properties into consideration. Among the second ones, though the scheme based on Hilbert curve can achieve better performance in terms of locality preservation properties and encoder efficiency, the rearrangement process of images changes the original correlation distribution, and the non-key pictures have at most two reference pictures when using single-view video coding structure for prediction. Additionally, the image characteristics of elemental images might be influenced by various pickup conditions, so these images are transformed to sub-images in some research.To limitation of the Hilbert-curve-based scheme, this paper presents a hierarchical prediction structure for effective sub-images coding, and a multi-threaded parallel implementation for the proposed structure. This hierarchical prediction structure is based on the hierarchical B pictures (HBP) structure for multi-view video coding. In this structure, the images are divided into key ones and non-key ones, and the images array is segmented into a two-dimensional (2D) array made up of groups of pictures (GOPs). The key pictures are intra-coded, and the non-key pictures are inter-coded using different modes according to their positions within the corresponding GOP. This paper introduces two terms"row hierarchy level"and"column hierarchy level"to represent the prediction dependencies in vertical and horizontal directions, respectively. In this structure, all rows are predicted by using only rows on a lower row hierarchy level as references and all columns are predicted by using only columns on a lower column hierarchy level as references. The experimental results show that employing the same coding strategy the proposed hierarchical prediction structure for sub-images coding can provide most images more reference pictures without changing the spatial position of each image, and thus it can achieve both higher image quality and better 3D effect.In order to improve the real-time performance of the coding scheme, a parallel implementation of the proposed hierarchical prediction structure is designed in this paper using multi-threaded architecture according to inter-row prediction dependencies. This scheme codes the GOPs located within the same column of the GOP array in parallel, within which, the rows are coded according to the ascending order of row level number, the rows with the same row level number are coded in parallel, and the sub-images located within the same row are coded according to the ascending order of column level number. The experimental results show that this parallel implementation can achieve lower coding delay, and thus effectively improve the real-time performance.