Continuous Surface Light Field Reconstruction From Non-Dense Sampling

With the development of stereo vision,virtual reality and 3D animation,the objects need to be reconstructed becomes more complication and rendering for the rebuilt objects requires more realistic display,So the high quality 3D modeling and rendering technique of complicated objects is one of the problems which are urgent works to handle in further research.In recent years,Image-based modeling and rendering technology get a lot of attention.Image-based modeling is aimed at obtaining information from the 2D image to restore the 3D object.Image-based rendering technology usual use more than one sample images to render new scenarios of arbitrary viewpoint,these became new researches and the development directions in computer graphics and computer visions field.Recently,light field technology get a lot of attention as one of Image-based modeling and rendering methods,it can get the dense images of 3D scene to restore the 3D scene or object and the rendering is independent of scene complexity,However,because of the camera angle and the space constraints of the imaging planes,the light field data based images have blurring phenomena during some depth range,So researchers introduce the concept of Surface Light Field(SLF)with the help of the 3D model.Modeling and rendering based on SLF utilize sample images of objects,reconstruct 3D model of object and realize the object's 3D model rendering under arbitrary view point.Usually sampling density will directly affect the precision of reconstruction and rendering,intensive sampling generally bring high precision reconstruction and rendering,but it needs the high cost of hardware,storing large amount of data and data redundancy,etc.Therefore,Researching continuous SLF construction based on getting images information under the condition of a non-dense sampling to render object under arbitrary view point,is a challenging problem.This paper research high accuracy 3D modeling and continuous SLF construction based on getting images information under the condition of a non-dense sampling,by light field rendering to verify the precision of continuous SLF.In terms of high precision 3D model reconstruction,we construct adaptive reference model,for unconstrained image set,improve the lack of existing model reconstruction methods,propose introduce normal gradient to restrict mesh deformation of reference model in photometric stereo reconstruction iterative process,to improve the reconstruction accuracy.In terms of constructing continuous SLF,we use spherical harmonics as the basis functions and improve the least squares fitting method.We take human face as an application example to verify the fitting of continuous SLF of 3D model under unconstrained non-dense sampling condition.The main innovation of our work summarized as follows:1?We proposed adaptive reference model construction method.Taking human face as an application example,based on multiple predefined facial model,we use weighted combination based local characteristics regional similarity and global similarity weighted fusion method to produce an adaptive reference model.The reference model can adapt the object and improve the quality of 3D face reconstruction.2?We proposed the method of 3D fine model reconstruction method based on photometric normal.Taking human face as an application example,we combined the technology of Laplacian deformation and photometric stereo technique and improved surface normal iterative process of photometric stereo reconstruction by enhancing constraints to photometric stereo-based normals to optimize the calculation of the normal vector,elaborating the high-quality 3d face reconstruction.3 ?We proposed use spherical harmonics as the basis functions to construct continuous SLF from non-dense sampling.Taking human face as an application example,we denote SLF as the linear combination of basis functions and select the spherical harmonics as the basis function,then we use the improved unconstrained least-squares fitting to construct continuous SLF.