Fundamental algorithms of space-variant vision: Non-uniform sampling, triangulation, and foveal scale-space

The architecture of the higher vertebrate visual system is both space-variant and multi-resolution. Space-variance, or foveal vision, refers to the increasing minimal angular resolution from the fovea to the periphery. Multi-resolution refers to the existence of multiple spatial frequency channels representing each region of the visual field. Computer vision historically has had a major commitment to multi-resolution computation. The application of foveal, or space-variant computer vision, has been much less explored, especially given the prominent role of foveal architecture in biological vision. The first reason for this is rooted in the technical difficulty of designing anti-aliasing filters for space-variant sampling. The second reason is the general lack of algorithms designed to work specifically on space-variant image formats.; Recent work in sampling theory has suggested a more modern approach than is provided by the historical approach associated with Shannon's sampling theorem. Briefly, Shannon's approach is to project a signal into the space of band-limited functions, using the sinc function as both basis and anti-aliasing filter. The more modern approach is to project into a Hilbert space with the criterion being minimal least-squared error. This allows the use of more general bases (e.g., polynomial splines, wavelets, etc.) and more general anti-aliasing filters. These advantages, together with Delaunay triangulation, are used to develop a general methodology for space-variant image processing. As a demonstration, several image processing algorithms in the space domain, such as boundary shape analysis and skin-color segmentation, have been adapted to this framework. The foveal pyramid is also developed and discussed as the image representation that facilitates template-matching and provides a basis for fast computation of the exponential chirp transform (Bonmassar & Schwartz, 1997). A further aspect of this work is the use of mesh based image representation, rather than the classical pixel based architecture. Future development of sensors and image architectures based on temporally asynchronous and spatially variant image representations are briefly discussed.