Research On Light Field Image Coding Based On LSTM Sparse Coding And CycleGAN Prediction

The light field image can collect the complete information of light in the three-dimensional space through the unique microlens array of the plenoptic camera.Therefore,high-dimensional information such as the scene depth can be extracted from the light field image.Such high-dimensional information can provide users with a sense of three-dimensionality and space immersion.The birth of light field images provides strong technical support for 3D reconstruction,depth estimation and virtual reality.However,due to the large amount of light information acquired by the light field technology,the light field image is huge,and a lot of unnecessary system resources will be wasted in storage and network transmission.Therefore,there is an urgent need for an effective light field image coding algorithm.In order to reduce the cost of using the light field image.This thesis will combine the structural characteristics of the light field image and the deep learning technology,and propose a new light field image coding framework that can effectively encode the light field image.The research work of this thesis is summarized as follows:Considering that a large amount of redundant information is mixed in a single light field image,which is not conducive to targeted processing,it is necessary to rearrange the pixels of the original light field image according to its viewpoint position to obtain the sub-aperture image array.The spatial redundancy information and viewpoint redundancy information in the field image are split into each sub-aperture image of the array.By observing the images in the array,the image content of a single sub-aperture image can express a large amount of scene information of the original light field image,and the image content between adjacent viewpoints shows extremely high corrclation.Therefore,when coding the original light field image,the sub-aperture image array can be split to two parts,and only one part of the sub-aperture image can be sparsely coded and transmitted,and the remaining uncoded sub-aperture images can be predicted.This thesis will combine deep learning technology to use an LSTM-based image coding model to sparsely code selected sub-aperture images to eliminate spatial redundant information in a single sub-aperture image.The prediction module is implemented by CycleGAN.It is used to simulate the generation of details between different viewpoints and the pixel shift of the main part.The prediction module based on CycleGAN can be used to eliminate the viewpoint redundancy contained in adjacent sub-aperture images.In the prediction process,the LSTM coded part is used as the reference module.Referring to the idea of intra prediction in H.264/AVC and H.265/HEVC,this thesis set 5 prediction modes for performing in different reference modules to choose the best predicted image.Under the idea of sparse coding and prediction,the proposed coding framework will be able to effectively reduce the bit rate and maintain excellent reconstruction quality.According to the experimental results,compared with the standard coding algorithms JPEG and JPEG 2000,the proposed coding framework has obvious advantages under the PSNR and MS-SSIM in low bit rate,and its MS-SSIM value can be equal to the mainstream algorithm HEVC,so the framework proposed in this thesis can complete the effective coding of light field images in the field of deep learning,and expand the new idea of light field image coding.