Research On The Plenoptic Sampling And Reconstruction In Large DOF Scenes

With the popularity of 3D movies and TVs,3D video technology has received extensive attention and great development.The current 3D movies and TVs only provide the specified view,which cannot satisfy users with a more real and intuitive 3D experience.In recent years,a large number of Multi-View Video(MVV)and Virtual Reality applications have been appeared,then users can obtain more fatanstic 3D experience by freely choosing an arbitrary view angle.In a word,MVV has become the main direction of multimedia field in the future.The premise of this application is how to effectively describe and perfectly reconstruct the scene.As an effective technology of scene description,Image-Based Rendering(IBR)has been widely researched for the low computational complexity and real visual experience.The current video coding standards such as Multi view video coding standard MVC and Three dimensional high efficiency video coding standard 3D-HEVC are the practical applications of IBR.Compared with the Model Based Rendering(MBR),IBR not only needs no/less depth information of the scene,but also possesses low computational complexity.However,in order to ensure the perfect reconstruction of the scene,more samples are needed.IBR is considered as a sampling and reconstruction problem.Firstly,the scene is sampled with finite set of images,and then the new viewpoint is rendered based on the obtained samples.Therefore,the key problem of the IBR is how to guarantee the perfect reconstruction of the scene with the minimum sampling rate.As the first parameter method of IBR is the Plenoptic Function,the sampling of the IBR is named as "Plenoptic Sampling".In this paper,we focus on the Plenoptic Sampling and its application in the following aspects.The first work of this dissertation is to extend the plenoptic sampling theory to the sampling and reconstruction in nonideal scenes.The previous plenoptic sampling theory only studies the scene without occlusion and non-Lambertian,and the corresponding research result it too ideal.By analyzing the essence of the spectrum of plenoptic function,for the first time,we separate the spectrum of the plenoptic function into two parts.The maximum and minimum depths of the scene build the main part,which conforms to the conclusion of previous plenoptic sampling theory.The second part is the broadening area introduced by the characteristics of the scene.According to the scene characteristics,we model the EPI of the complex scenes,and analyze the extended spectrum.We obtain the quantitative sampling rates and the corresponding optimal reconstruction filters for the non-Lambertian reflection scenes,the occlusive scenes and the slantled plane scenes.The conclusion of this paper extends the previous plenoptic sampling theory,which provides a theoretical support for the application of sampling theory to the real scene.The second work of this dissertation is the sampling rate research in effective Depth of Field(DOF)scenes.Camera brings the defocus problem,and similar phenomenon exists in the process of the reconstruction filter with constant depth.Thus,the further research of sampling with constant depth filter is needed.In this paper,the effective DOF is determined by the maximum acceptable circle of confusion,which makes sure the scene is captured in focus with ordinary camera.Then,we analyze the reconstructed distortion model of constant depth filter.By introducing the maximum acceptable circle of confusion in the reconstruction process,we obtain the new sampling rate,which guarantees the reconstruction without defocus.Furthermore,a method with optimal plenoptic sampling is obtained.By considering the camera parameters,the effective DOF is determined,if the actual effective DOF cannot be satisfied,adjust the camera parameters or used the layer based model to sampling the scene.The camera parameters such as resolution,focal length,and aperture diameter are the key factors that restrict the sampling process,and these factors make it possible for sampling without defocus in the limited camera sampling conditions.Finally,a depth layer model based sampling and reconstruction framework for scenes with large DOF is proposed.With this framework,the plenoptic sampling theory can be applied into practical applications.Generally,two problems exist in practical applications of the plenoptic sampling.The first one is the defocus in the sampling,and the second one is that the sampling rate is always too large to apply in the real scenes.Therefore,a framework with layer based is presented.By separating the scene,the complex scene with large DOF is divided into multiple depth layers with effective DOF.Each layer is determined by the maximum acceptable circle of confusion,which ensures the focus sampling in the layer.Then,through the geometrical analysis of the reconstruction filter with constant depth,the new sampling rate is obtained on the premise of the reconstruction without defocus.With this layer based sampling framework,the complex with large DOF is sampled with the high frequency information of the scenes saved.By using the focus region recognition method based on color consistency,the focus area of each layer is obtained and then combined to achieve the final all-in-focus views.The sampling framework in this paper overcomes the two difficult problems in the practical application of the plenoptic sampling theory that mentioned above.Further,by capturing the real scene with both ordinary camera and plenoptic camera,we get the sampling and reconstruction result of real scenes.In general,the dissertation firstly extends the plenoptic sampling theory,and analyzes the sampling and reconstruction of the non-Lambertian reflection scenes,the occlusive scenes and the slanted plane scenes.Secondly,considering the application of the plenoptic sampling theory,the dissertation studies the sampling without defocus in effective DOF scenes,and proposes a depth layer model based sampling and reconstruction framework for scenes with large DOF.Both ordinary camera and plenoptic camera can be used to capturing the real scene with this framework.The plenoptic sampling and reconstruction research in this disertation can be widely used in the filed of MVV,which provides the research and application for MVV data acquisition.