| Conventional video technology has a limitation that the viewpoint is fixed by the camera that captures the scene. Free viewpoint video (FVV) is an innovation to conventional video by allowing the user to choose his/her viewpoint across a scene. It has a variety of applications in interactive media, 3D TV, and the filming and gaming industry. This dissertation proposes a practical FVV system which uses Image-based Rendering (IBR) technology to specify and render a new view based on only a few input images/videos. Key components for this system are addressed and presented with experimental results.;To deploy existing IBR method, the multiple cameras that capture the scene must be calibrated to obtain a set of parameters with a strict procedure before use. To remove this limitation, a practical virtual view specification approach that allows the cameras to be uncalibrated is proposed, together with the corresponding techniques to synthesize the new view. A complete system is implemented with a multiple camera setting to show the feasibility and effectiveness of proposed approaches.;Further discussion reveals that the virtual view synthesis problem can be viewed as Bayesian inference and improvement can be gained by exploiting the statistical property of natural images and the strong correlation among multiple input images. A joint conditional random field (CRF) framework based on multiple views and an on-line learning algorithm is presented and verified by experiments.;Finally, it shows that this joint conditional random field can also be applied to a classic computer vision problem, motion estimation, to effectively use information from multiple images and improve the results. |
