Geometry And Appearance Reconstruction Method Under Natural Environment

Digitalization of real objects is a fundamental research area in computer graph-ics and computer vision.The problem of recovering high-quality 3D geometry and reflectance with simple device setup has a high research value and a variety of applica-tions in manufacturing,heritage preservation,special effects,augmented reality,etc.The image of an object is determined by its geometry,reflectance properties,and the environmental lighting.Existing works usually solve them separately,impose strong constraints on geometry,reflectance and lighting,or use special capture devices,which cannot be used as general solutions for geometry and reflectance recovery that apply to natural environment or objects with general materials.In this dissertation,we propose a reconstruction algorithm that jointly recovers both geometry and appearance from a continuous video sequence input.The algorithm robustly recovers a normal volume and iteratively optimizes on object shape,appear-ance and environment lighting based on the key observation that normal,appearance and lighting affect the temporal trace differently.The algorithm simplifies the capture process and the device setup of geometry and appearance recovery.To further extend the proposed algorithm to scenarios where the dense input is not available,we propose an algorithm with sparse image input.We regularize the problem by adding a sparse constraint on appearance.And we have solved two sub-problems of geometry and appearance reconstruction-recovering appearance and lighting with known geometry,and recovering appearance and geometry with known lighting.These two sub-problems are important steps of geometry and appearance recovery algorithm under unknown lighting.For the sub-problem that recovers appearance and lighting with known geometry from sparse input,we solve lighting and appearance by estimating low frequency and high frequency components of lighting and appearance separately.The ambiguity of lighting and appearance is well handled by the spectral separation.For the sub-problem that recovers appearance and geometry with known lighting,we iteratively optimize geometry and appearance by using a specular-compatible feature point matching method and a single-frame normal and appearance recovery algorithm.The solution can apply to most common cases,and it further simplifies the capture process.Our work proves the practical possibility of joint geometry and appearance recov-ery under unknown uncontrolled natural lighting environment.Since our methods do not need active lighting or specialized devices,they are broadly applicable to a large va-riety of practical scenarios.We hope this work can further develop the recovery methods in related industries and make object digitalization more accessible to the public.