On three-dimensional spacetime oriented energy representation for spatiotemporal stereo and motion recovery

This dissertation presents novel approaches to recovering temporally coherent estimates of 3D structure and 3D flow maps of a dynamic scene from a sequence of binocular stereo images The proposed approaches are based on matching spatiotemporal orientation distributions and their approximation in the form of spatiotemporal quadric elements (stequels) between left and right temporal image streams, which encapsulate both local spatial and temporal structure for disparity estimation. By capturing spatial and temporal structure in this unified fashion, both sources of information combine to yield disparity estimates that are naturally temporal coherent, while helping to resolve matches that might be ambiguous when either source is considered alone. Also, by allowing subsets of the orientation measurements to support different disparity estimates, an approach to recovering multilayer disparity from spacetime stereo is realized. Moreover, the matched primitives allow for direct formulation of instantaneous 3D flow vector measurements. The proposed methods have been implemented with real-time performance on commodity GPUs. Empirical evaluation shows that the approaches yield qualitatively and quantitatively superior disparity estimates in comparison to various alternative methods, including the ability to provide accurate multilayer estimates in the presence of (semi)transparent and specular surfaces and recover dense 3D flow estimates accurately and efficiently.