Research On Key Technology For Accurate Reconstruction Of 3D Light Field

Three-dimensional(3D) display technology can provide people with natural 3D visual percep-tion,which is in line with human observation of the real world.Among all 3D display technologies,light field display has attracted wide attention due to its superior display quality,which is mainly due to its great power to reconstruct light field of scenes accurately,thus being able to support all kinds of depth cues required by comfortable 3D visual experience.However,quality of current light field display is still limited by the hardware system's reconstruction accuracy of light rays and the resolution of display contents.Therefore,research on the technology of high-precision light field reconstruction is of great significance to improve the display quality.We first study high-precision light field reconstruction in real space.Due to the huge complex-ity of the hardware composition of multi-projection system,assembly errors and image distortions are hard to avoid,leading to deviations between the actual displayed light field and the ideal one.Then the system will fail in providing right 3D visual perception.To solve this problem,we pro-pose a calibration method to improve accuracy of the displayed light field.We first use randomly placed cameras without the need of calibration to sample the distorted projection light field from multiple positions.Then we use Bezier surface to represent the complex light field distortion,in order to jointly calibrate errors caused by system assembly and optical distortion.The experiment results prove that our method can calibrate the large-scale dense light field at ray level and obvi-ously improve the display quality.Although this method is proposed based on a multi-projection system,in principle it is not limited by structure of the display system and can provide ideas for subsequent research on light field calibration.Next,we research on synthesis of high-resolution light field display contents,aiming to im-prove display quality from the perspective of image source.Restricted by capture hardware,den-sity and accuracy of current captured light field cannot meet the needs of existing display systems.Considering that it is easier to obtain spatially high-resolution light field images with existing equipment,we focus on study angular light field super-resolution and propose two kinds of dense view synthesis methods.The first one mainly processes light field obtained by planar acquisition array.By utilizing disparity as geometry proxy,we reduce the density requirement for input views.Then by introducing occlusion network,we improve the geometric estimation accuracy and inpaint holes reasonably in the generated novel views.Finally,cycle consistency check is used to enable unsupervised learning,which further breaks the hardware limitation.Quality of views generated by our method is comparable to supervised methods on common image evaluation metrics and is even superior around occlusion regions with sharper object edges,which also proves precision of the synthesized light field.However,the method above is only suitable for light field capture systems with planar camera array and display systems with planar viewpoint distribution.In order to remove such restrictions,we propose a free-viewpoint synthesis method,which can take casually captured sparse views as input and synthesize novel views to meet view density and distribution requirement of various display systems.To process sparse inputs,we build our depth estimaiton network on state-of-the-art multiview stereo systems.To improve accuracy of estimated geometry and visual quality of synthesized views,we introduce attention mechanism and use it to mask out visible pixels from input views at target viewpoint,which can further ensure the pixels used for depth inference and view blending obeys the assumption of photometric consistency.This will guarantee stability and consistency of scene structure in the generated views.The experimental results validate that the synthesized view sequence can reproduce high-quality light field in space when used as the input of display system,and can provide more realistic appearance than computer-generated 3D models.Finally,we summarize the research contents of this paper,and look forward to the research directions that can be further explored in the future.