| What is the object in this picture? What would this object look like in a different scene? These questions are the core of computer vision (image analysis) and computer graphics (image synthesis). They are traditionally approached through specific problems like matting and composition, image segmentation, capture of high-speed video, depth-from-defocus, and visualization of multi-spectral video. For the solutions to these problems, both the inputs and outputs are frequently images. However, they are not the same kind of images as photographs, but a more general form of 2D maps over the surface of an object, lens, or display. For example, an image describing reflectivity over a surface is known as a texture map, an image composed of subimages with varying optical center is a light field slab, and an image describing the partial coverage of a backdrop by an object is a matte.;This dissertation introduces multi-parameter video, a framework for describing generalized 2D images over time that contain multiple samples at each pixel. Part of this framework is diagraming system called optical splitting trees for describing single-axis, multi-parameter, lens (SAMPL) camera systems that can be used to capture multi-parameter video. My thesis is that a SAMPL is a generic framework for graphics and vision data capture; that multi-parameter video can be accurately and efficiently captured by it; and that algorithms using this video as input can solve interesting graphics and vision problems. In defense of this thesis I demonstrate physical SAMPL camera hardware, many registered multi-parameter video streams captured with this system, and a series of problems solved using these videos. The leading problem I solve is the previously open unassisted video matting/subpixel object segmentation problem for complex, dynamic, and unknown backgrounds. To show generality I also demonstrate high speed video capture, multi-modal video fusion, multi-focus video fusion, and high dynamic range video capture. |
