A COMPUTATIONAL THEORY OF SPATIO-TEMPORAL AGGREGATION FOR VISUAL ANALYSIS OF OBJECTS IN DYNAMIC ENVIRONMENTS

A theory of spatio-temporal aggregation is introduced as an explanation of what it is possible to compute at an early stage in the visual analysis of objects. Spatio-temporal aggregation is defined as the process of grouping together elements in an image sequence whose motions and positions have consistent interpretations as the retinal projections of a coherent or isolated cluster of 'particles' in the physical world. The information abstracted by spatio-temporal aggregation represents early decisions about which distinct elements belong to the same physical entities or object fragments and estimations of the motions of those fragments relative to the observer. The assumptions of confluence and adjacency are made in order to constrain the infinity of possible interpretations to a computationally more manageable domain of plausible interpretations. It is shown that confluence and adjacency lead to the derivation of specific rules for grouping which permit the appropriate aggregation of rigid and quasi-rigid objects in motion and at rest under a variety of conditions. The theory is reconciled with existing computational theories of vision so as to complement them, and to provide a useful link in the continual abstraction of visual information.