Novel view synthesis using quasi-sparse feature correspondences

The problem of novel view synthesis can be stated as follows; given a set of photographs taken from known positions, generate a new rendering of the scene. This dissertation presents several such techniques that differ in applicability based on complexity of the scene, the geometry of the camera positions, and whether prior knowledge of the objects' geometry is available.; The first technique was inspired by the FACADE System developed at U. C. Berkeley by Taylor and Debevec. It assumes that we are given a single photograph taken from an unknown position and orientation, that the objects' shapes are given by linearly parameterized models, and that the camera's focal length is unknown.; We then present a series of techniques that employ Epipolar Plane Images (EPIs). EPI analysis is used to obtain a quasi-sparse set of feature correspondences. We say that the set is quasi-sparse because it corresponds roughly to the set of image edge points. This set is far denser than the set of corners in the image and much sparser than the entire set of pixels.; In the first method, we are able to use these correspondences to render new views of a scene from a series of pictures taken from known positions along a linear trajectory. This method explicitly avoids performing a full 3-D reconstruction of the scene. These results can be improved by using a two-dimensional set of camera positions, i.e. moving the camera along a plane and keeping the optical axis perpendicular to that plane. A simple technique that employs these type of images is given.; Still using camera motion along two axes, we exhibit a technique for rendering new views of a “2½-D” scene, and finally we present a technique for view synthesis of general three dimensional scenes. This procedure is based on selective depth determination combined with basic ray-tracing methods. Furthermore, many of the calculations required can be done by directly using graphics hardware, and are therefore very efficient.