Compression and interactive streaming of light fields

Remote interactive viewing of high-quality, photo-realistic 3-D objects and scenes has the potential to enable new applications in virtual reality, gaming, virtual museums and e-commerce. Image-based rendering which is used for interactive photo-realistic viewing, however, relies on a large amount of image data. This makes remote access challenging. This thesis makes contributions towards enabling the interactive remote viewing of a large image-based rendering dataset called a light field.; An interactive light field streaming system that accounts for latency as experienced by the user, and incorporates prediction of the user's future view trajectory, is presented. A key component of this is a method for rate-distortion optimized packet scheduling for streaming encoded image data. This system selects and transmits the appropriate image data to the user based on what the user desires to view. In doing so, it considers the constraints of the network, while optimizing for best visual image quality.; A theoretical framework for analyzing the rate-distortion performance of the entire light field coding, rendering and streaming system is proposed. Simulation results based on the theoretical model show good correspondence with actual experimental results. This theory allows better understanding the parameters of the system, and can potentially be helpful in optimizing the performance of the system.; An encoding of the light field image data that attempts to give better rate-distortion streaming performance is proposed. This encoding which uses multiple representations of each image, and is shown for certain user view trajectories and datasets to improve performance over conventional coding techniques.