On the Role of Representation in the Analysis of Visual Spacetim

The problems under consideration in this dissertation centre around the reprepresentation of visual spacetime, i.e., (visual) image intensity (irradiance) as a function of two-dimensional spatial position and time. In particular, the over-arching goal is to establish a unified approach to representation and analysis of temporal image dynamics that is broadly applicable to the diverse phenomena in the natural world as captured in two-dimensional intensity images. Previous research largely has approached the analysis of visual dynamics by appealing to representations based on image motion. Although of obvious importance, motion represents a particular instance of the myriad spatiotemporal patterns observed in image data. A generative model centred on the concept of spacetime orientation is proposed. This model provides a unified framework for understanding a broad set of important spacetime patterns. As a consequence of this analysis, two new classes of patterns are distinguished that have previously not been considered directly in terms of their constituent spacetime oriented structure, namely multiplicative motions (e.g., translucency) and stochastic-related phenomena (e.g., wavy water and windblown vegetation). Motivated by this analysis, a representation is proposed that systematically exposes the structure of visual spacetime in terms of local, spacetime orientation. The power of this representation is demonstrated in the context of the following four fundamental challenges in computer vision: (i) spacetime texture recognition, (ii) spacetime grouping, (iii) local boundary detection and (iv) the detection and spatiotemporal localization of an action in a video stream.