A Novel View Synthesis Rendering Algorithm Based On Deep Learning

Novel view synthesis(NVS)generates images from unseen viewpoints based on a set of input images of a scene.The value of NVS problem lies in creating a realistic virtual scene based on a few input images so that it will eventually allow users to navigate through the virtual scene freely by predicting the rendered images observed from novel viewpoints.The techniques and applications of NVS mainly involve computer graphics,computer vision,3D geometry modeling,and virtual reality.It is a challenge in both computer graphics and vision communities because of inaccurate lighting optimization and geometry inference.Although current neural rendering methods have made significant progress,they still struggle to reconstruct global illumination effects like reflections and exhibit ambiguous blurs in highly view-dependent areas.This work addresses high-quality view synthesis to emphasize reflection on nonconcave surfaces.This thesis proposes Deep Flow Rendering(DFR)that optimizes direct and indirect lighting separately,leveraging texture mapping,appearance flow,and neural rendering.A learnable texture is used to predict view-independent features,meanwhile enabling efficient reflection extraction.To accurately fit view-dependent effects,a constrained neural flow is adopted to transfer image-space features from nearby views to the target view in an edge-preserving manner.Then a fusing renderer is further implemented that utilizes the predictions of both layers to form the output image.The experiments demonstrate that our method outperforms the state-of-the-art methods at synthesizing various scenes with challenging reflection effects.