A discriminant hypothesis for visual saliency: Computational principles, biological plausibility and applications in computer vision

It has long been known that visual attention and saliency mechanisms play an important role in human visual perception. However, there have been no computational principles that could explain the fundamental properties of biological visual saliency. In this thesis, we propose, and study the plausibility of a novel principle for human visual saliency, which we denote as discriminant saliency hypothesis. The hypothesis states that all saliency decisions are optimal in a decision-theoretic sense. Under this formulation, optimality is defined in the minimum probability of error sense, under a constraint of computational parsimony. The discriminant saliency hypothesis naturally adapts to both stimulus-driven (bottom-up) and goal-driven (top-down) saliency problems, for which we derive the optimal discriminant saliency detectors, in an information-theoretic sense. Statistical properties of natural stimuli are also exploited in the derivation for the constraint of computational parsimony.;To study the biological plausibility of discriminant saliency, we show that, under the assumption that saliency is driven by linear filtering, the computations of discriminant saliency are completely consistent with the standard neural architecture in the primary visual cortex (V1). The discriminant saliency detectors are also applied to the set of classical displays, used in the study of human saliency behaviors, and shown to explain both qualitative and quantitative properties of human saliency. These results not only justify the biological plausibility of the discriminant hypothesis for saliency, but also offer explanations to the neural organization of perceptual systems. For example, we show that the basic neural structures in V1 are capable of computing the fundamental operations of statistical inference, e.g., assessment of probabilities, implementation of decision rules, and feature selection.;Finally, we evaluate the performance of the derived discriminant saliency detectors for computer vision problems. In particular, we apply the top-down saliency detector to the problem of weakly supervised learning for object recognition, and show that the detector outperforms the state-of-the-art saliency detectors in (1) capturing important information for object recognition tasks, (2) accurately localizing objects of interest from image clutter, (3) providing stable salient locations with respect to various geometric and photometric transformations, and (4) adapting to diverse visual attributes for saliency. We then evaluate the performance of the bottom-up discriminant saliency detector in the applications where no recognition is defined. In particular, we show that the bottom-up discriminant saliency implementation accurately predicts human eye fixation locations on natural scenes. In another application of discriminant saliency, we discuss a Bayesian framework to integrate top-down and bottom-up saliency outputs, where the top-down saliency is interpreted as a focus-of-attention mechanism. Experimental results show that this framework combines the selectivity of the top-down saliency with the localization ability of the bottom-up interest point detectors, and improves the object recognition performance.;Overall, the excellent performance of discriminant saliency in both biological and computer vision evaluations justifies the plausibility of discriminant hypothesis as an explanation for human visual saliency.