Appearance And Geometry Modeling With Low-Dimensional Data

Appearance and geometry data is the fundamental representation of 3D objects and scenes in computer graphics and is widely used in many applications such as games,movies,augment reality,etc.On one hand,directly acquisition of appear-ance and geometry data is very hard due to its high-dimensional nature;on the other hand,traditional image and video sequences is only a low-dimensional projection of high-dimensional appearance and geometry data under certain lighting conditions and viewpoints.Thus,efficiently modeling high-dimensional appearance and geometry data with low-dimensional image observations is still a challenge problem in computer graphics.This dissertation proposed several methods for efficient modeling appearance and geometry from low-dimensional image observations.To tackle this challenge problem,the key idea is to efficiently exploit rendering projection between high-dimensional ap-pearance and geometry data and its image observations.The maj or contributions of this dissertation are:· An adaptive lighting method for capturing SVBRDFs of piecewise materials is proposed.The proposed method can efficiently reconstruct SVBRDFs of planar materials from two image observations,by optimizing a good lighting pattern as best rendering projection according to material spatial distribution.Compar-ing with traditional methods,the proposed method reduces numbers of input for SVBRDF capturing and imporves capturing efficiency.· A single image appearance modeling method based on self-augmented convolu-tional nerual networks is proposed.To overcome insufficient number of labeled data,the proposed method introduced rendering as part of network training pro-cess;during training,additional labeled data is generated on-the-fly with current network and rendering from large collections of unlabled data.The trained net,work is able to estimate plausible SVBRDF results from a single photograph,greatly accelerates the appearance modeling pipeline.· A geometry and appearance generation method based on multiple projection GAN framework is proposed.By using multiple differentiable rendering pro-jections,the proposed framework enables directly modeling geometry and ap-pearance distributions from its low-dimensional image collections.For geometry generation,the proposed framework further improves generation quality by intro-duce view prediction network and iterative training between generation network and view prediction network.The proposed method significantly reduced label-ing effort for training geometry and appearance generative models,enabled novel solutions for shape and appearance generation.