Study On Novel View Synthesis

The problem of Novel View Synthesis is an import problem in the combined field of computer graphics and computer vision. That is: given multiple images of a 3D scene captured by a set of cameras, or by a single moving camera, generate a synthetic view of the scene, as it would appear from a new viewpoint. Such new views can be used in teleconferencing or in 3-dimensionalizing monocular film footage. Algorithms to solve this problem can be subdivided into two categories: scene reconstruction and image-based rendering.This thesis studies on Novel View Synthesis mainly in two aspects: real-time Novel View Synthesis algorithm and high precision Novel View Synthesis algorithm. Study on real-time Novel View Synthesis comes from the novel view synthesis part of the project of Shanghai Jiao Tong University collaborated with Omron company. And then further, we do some research on high precision Novel View Synthesis algorithm.As for real-time Novel View Synthesis algorithm, we investigate two IBR methods: view morphing and algorithm based on trilinear tensor. These two methods take the uncalibrated images as input. They can both get the results without obtain the 3D model of the scene. Feature match is the foundation of the methods using uncalibrated images. In this thesis,we use SIFT feature match algorithm and enhance it by eliminating the duplicate match. The experiment results of view morphing show that this algorithm can get good results on the whole in real-time.As for high precision Novel View Synthesis algorithm, the thesis mainly focuses on the algorithm under the energy minimization framework. This method used in this problem is aroused from that used in 3D reconstruction. Our methods take the images and the corresponding project matrix as input and such error can be avoided consequently. In addition, the energy minimization algorithm based on graph cuts turn out to be the best not only on the speed but also on the accuracy in this thesis.