Saliency, attention and visual search: An information theoretic approach

This dissertation explores the concept of visual saliency as it pertains to attentional selection, visual search, and machine vision. A novel framework for visual saliency is put forth derived from consideration of the problem in the context of information theory. The proposed definition is distinguished from previous efforts on this front and is demonstrated to be a natural principled definition for salient visual content. Specifically, the proposal deemed Attention by Information Maximization (AIM) seeks to select visual content that is most informative in a formal sense in the context of a specific scene, and is put forth in a form that is amenable to considering more general definitions of context. Efficacy in predicting human gaze patterns is demonstrated and the proposal is revealed to outperform existing models in the prediction of fixation points. With regard to biological plausibility, an important consideration is the extent to which the model behavior agrees with the psychophysics and neurophysiology literature. To this end it is revealed that AIM is able to account for an unprecedented range of classic psychophysics results including some subtle and counterintuitive results and may be achieved via a neural implementation that is consistent with observations concerning surround modulation in the cortex. More general modeling considerations are also addressed including compatibility with descriptions of how attention as a whole is achieved and constraints on possible architectures for achieving attentional selection in light of recent psychophysics and neural imaging results. The applicability of this definition within a machine vision context is also discussed revealing some interesting properties as emergent from the basic framework.