Modeling the design of visual search tasks in human-computer interaction

Previous studies in identifying factors related to search efficiency resulted in empirical models that are hard to apply. In order to overcome the limitations inherent in descriptive empirical methods, a quantitative model of visual search, the Guided Search (GS) model, which has been validated in many perceptual tasks, was extended to Human-Computer Interaction (HCI) settings. By testing the validity of the GS model in HCI visual search tasks, it provided insights in the perceptual factors related to the reduction of search time for HCI visual search tasks. First, GS simulation models were separately defined for a menu and an icon search task. Model parameters were determined by obtaining the best fit between model predictions and experimental data from 300 menu search trials and 300 icon search trials with 10 subjects respectively. The validity of GS models in HCI visual search tasks was examined in a series of three experiments with 40 subjects. In these experiments, the GS models were used to generate comparative menu and icon designs based on pre-defined search frequency using a Simulated Annealing (SA) optimization method. Two sets of performance comparison were made: one was between two designs generated by the GS model, the other was between a design generated by the model and a design generated by a human designer. The experimental results indicated that key perceptual aspects of a menu search task could be captured and manipulated by the GS model to minimize search time when cognitive factors were kept constant. However, the GS model by itself was not sufficient to generate designs as good as those made by a human because it was not able to account for cognitive factors such as memory search, which proved to be important in determining search efficiency in the menu search task. The GS model was not as successful in capturing the perceptual factors for an icon search task due to the complexity of the visual appearance of the icons. The GS models could be further developed into tools to evaluate or automate designs for HCI visual search tasks.