Research On Coding Method Of Light Field Dense Sub-aperture Image

Light field imaging can simultaneously capture the position and direction information of the light rays in space,so the digital refocus,full depth-of-field extension and editable stereoscopic imaging,functions that inaccessible for conventional cameras,can be achieved by light field imaging.Furthermore,since light field image contains continuous and slight motion disparity,it can supply more real and natural viewing experience and reduce visual fatigue.Therefore,under the background of that the current public urgently needs novelty and immersion for media content,light field technology will become the key support for the new generation of visual imaging technique and lead the renovation for the future stereo display system.Light field dense Sub-Aperture Image(SAI)has become the research focus of computational photography,but its own huge amount poses a great challenge for data storage and network transmission,and even restricts the development and practical application of light field technology.For this reason,it is the key point of breaking the light field technological bottleneck to study efficient compression of light field dense SAI.The existing light field compression approaches mostly regard the pseudo sequence of SAI as the object,but they do not take the angular correlation and sparsity of light field into account.As a result,there is much room for reducing the redundancy in light field data.To improve the coding efficiency of light field dense SAI,based on the idea of sparse coding and reconstruction after decoding,this thesis carries out the research in terms of building the framework of sparse coding and reconstruction after decoding,proposing the method of depth estimation and optimization,as well as the scheme of sparse optimization and residual enhancement.The main researches and contributions are demonstrated in the following:The large number of light field SAI limits the efficiency of encoder.For this issue,we utilize the angular correlation among the SAIs to build the framework of light field dense SAI sparse coding and reconstruction after decoding.This framework constructs an Epipolar Plane Image(EPI)based epipolar line slope matching cost function.Based on the light field two-plane parameterization model,we use the slope of epipolar line computed by this function to derive the estimated depth value of point,and finally generate the corresponding depth map.High Efficiency Video Coding(HEVC)based multi-view video plus depth coding structure can code a small number of viewpoints and reconstruct more viewpoints at the decoder.To this end,we sparsely select a part of SAIs and their corresponding depth maps,and convert them into multi-view pseudo sequences.Moreover,the multi-view video plus depth codec is employed to encode the multi-view pseudo sequences,and the Depth Image Based Rendering(DIBR)technique is adopted to reconstruct the SAIs at the decoder.The experimental results demonstrate that our proposed framework can yield the comparatively precise depth map,so that the high-quality SAI can be reconstructed.Consequently,compared with the traditional SAI coding methods,our proposed method can significantly decrease the light field dense SAI bitrates,and improve the coding efficiency.The reconstructed SAI quality depends on the accuracy of depth map.To further improve the light field dense SAI coding performance,we propose a depth estimation and optimization method.According to the vignetting artifacts,we compute two depth maps from two directions EPIs and design a depth merging algorithm based on the position in angular domain.However,the slope of epipolar line located in the low frequency region is insignificant,so it leads to inaccurate depth estimation and SAI reconstruction.Due to it has the same pixel variation tendency in the low frequency region between the texture image and its associated depth map,we design a weighted filter,where the weight factor is determined by the pixel variation tendency of SAI,for depth denoising.Additionally,we decide a reasonable number of iterations for depth optimization through analyzing the results of the quadratic polynomial interpolation iteration algorithm.The experimental results indicate that our proposed method enhance the depth map compared to the typical light field image depth estimation algorithm.Therefore,the light field dense SAI sparse coding approach based on this method outperforms other light field coding approaches.The accurate depth map induces the high-quality reconstructed SAI.To further decrease the number of to-be-coded SAI for higher coding efficiency,we propose a light field dense SAI coding scheme of sparse optimization and residual enhancement.This scheme tries all kinds of the combinations of SAI sparse sampling steps and depth computational steps.A reasonable step combination is decided via reconstruction performance and depth computational time analysis.In order to exploit more relationship among the SAIs,this scheme proposes a content-similarity-based frame arrangement algorithm,which can adjust the reference order for the pseudo sequence in accordance with the similarity between two SAIs.To compensate the reconstruction distortion caused by longer SAI sparse sampling step,this scheme obtains the residual information between the original SAI and its reconstruction before coding,and enhances the reconstruction using the residual information.The experimental results manifest that the content-similarity-based frame arrangement algorithm contributes to the improvement of multi-view pseudo sequence coding performance.In addition,although the residual information increases bitrates,the enhancement for reconstruction makes this scheme better than other light field image sparse coding.