Research On Virtual Viewpoint Generation Technology Based On Computational Plenoptic Function

Human live in the world of three-dimensional,and synergistic effect of eye and brain enables people to perceive three-dimensional environment and objects.But the traditional two-dimensional graphics and display technology are limit of human perception in the two-dimensional world,a loss of real world caused in the transmission and display of the process.In the perception of human of the full three-dimensional world,stereoscopic display technology and virtual reality technology appear,and become the research hotspot.The acquisition of 3D scene information is limited by the existing hardware condition storage technology and so on.It can not fully collect information from different viewpoints in 3D scene,and virtual viewpoint technology arises at the historic moment.Virtual view generation technology,which can recover certain information from 3D scene by using a collection of samples,has become the core issue of hot research areas.It has been widely applied in auto-stereoscopic display technology and virtual reality technology and so on.In different applications,how to express the scene,how to render scene in real-time as the number of viewpoint increase,become a difficult problem of virtual viewpoint generation.Traditional method of virtual view generation in image based rendering is based on the study of how to use the known reference view to directly generate the corresponding virtual view images.It mainly depend on the stereo matching,depth estimation.This method has been widely applied in a certain extent,but there are problems of computational complexity,and also the number of view generated is less intensive.This thesis will be based on image rendering and combining with the research and analysis of plenoptic function.The key problem of IBR technology is the virtual viewpoint generation.So we research it from the view of plenoptic function,and consider it as a reinterpretation of the plenoptic function.This thesis will research on the acquisition,reconstruction and resampling of the plenoptic function,which is the key technology in plenoptic function.In this thesis,we first analyze the expression models of all optical functions under multiple dimensions,and select two dimensional panoramic mosaic technology and four-dimensional light field technology.In this thesis,the fish eye lens image is used to collect the panoramic image,and the image distortion is corrected by two ways.One is the method of calibration based on camera parameters,and the other is based on the assumed imaging model,and the algorithm is used to correct the distortion of the fish eye image.We use SIFT for image feature extraction,and using the BBF algorithm and RANSAC algorithm for feature point matching and purification,so as to calculate the camera's pose.We use Bundle Adjustment to realize camera parameter adjustment,and use the image fusion algorithm to get the final panorama image.Thus,we completed the reconstruction of 2D plenoptic function.In addition,a virtual reality system based on panoramic roaming is proposed and implemented in this thesis.Meanwhile,based on four-dimensional all optical function,we propose a free viewpoint generation algorithm based on pixel mapping and interpolation.We use bilinear interpolation algorithm,CUDA and Direct 3D are used for parallel processing and realize the real-time light field rendering based on GPU.