Accurate real-time reconstruction of distant scenes using computer vision: The recursive Multi-Frame Planar Parallax algorithm

In this dissertation, we detail the Recursive Multi-Frame Planar Parallax (RMFPP) algorithm, a recursive extension of Irani et al.'s Multi-Frame Planar Parallax (MFPP) batch algorithm that allows real-time reconstruction of distant static scenes using computer vision, with expected error that increases only linearly with depth. We present an overview and comprehensive derivation of the theoretical foundation on which the RMFPP algorithm is built, including the seminal planar-parallax work by Sawhney. We derive a recursive cost function that preserves more of the problem's nonlinearity than does the cost function in the MFPP algorithm, which allows a more accurate recursive procedure. In order to obtain a recursive algorithm, we remove the geometry-refining optimization that is present in the MFPP algorithm; however, we empirically show that our algorithm degrades gracefully in the presence of geometric error. We present results using both synthetic and real imagery that show that the RMFPP algorithm is at least as accurate as the original MFPP batch algorithm in many circumstances, is preferred to both fixed- and dynamic-baseline two-frame methods, and is suitable for real-time use.