Modeling the control of attention by visual and semantic factors in real-world scenes

Recently, there has been great interest among vision researchers in developing computational models that predict the distribution of saccadic eye movements in various visual tasks. The analysis of attentional guidance by low-level visual features in real- world scenes is difficult because the strength and distribution of visual saliency cannot be controlled. Moreover, real-world scenes are composed of objects with not only low-level visual features such as shape and color, but also high-level features such as meaning and semantic relations among them. While it has been found that low-level visual features guide eye movements during visual tasks in both stimulus-driven (bottom-up) and goal- driven (top-down) ways, the influence of high-level features such as meaning and semantic relations among scene objects on eye movements in such situations is still unknown.;In this manuscript, I propose a top-down model of visual attention which is based on visual similarity between the target and regions of the search scene using a histogram- matching technique, where similarity is defined for several feature dimensions such as color, orientation, or spatial frequency. The amount of attentional guidance across visual feature dimensions is predicted by a measure of informativeness of each feature dimension. Then, using a novel eye-movement analysis on visual saliency maps, I will show the dynamics between top-down and bottom-up guidance, in which top-down control is initially weak but quickly dominates search while narrowing its focus, whereas bottom-up control is slowly diminishing while using a constantly large visual span. Moreover, in order to build a foundation for a comprehensive, general computational model that also accounts for higher-level attentional guidance, I will show the existence of semantic guidance effects on eye movements, that is, subjects' tendencies of consecutively fixating semantically similar objects during scene inspection and fixating objects that are semantically similar to the search target during scene search. Semantic similarity of objects will be computed by applying Latent Semantic Analysis (LSA) to object descriptions in real-world scenes.;The results of my doctoral thesis will broaden our understanding of attentional guidance in real-world scenes under various visual tasks and should eventually lead to a general computational model of attentional guidance.