THE USER'S MENTAL MODEL OF AN INFORMATION RETRIEVAL SYSTEM: EFFECTS ON PERFORMANCE

An empirical study was performed to train naive subjects in the use of a prototype Boolean logic-based information retrieval system on a database of bibliographic records drawn from the OCLC system. The research was based on the mental models theory from cognitive psychology which proposes that people can be trained to develop a "mental model" or a qualitative simulation of a system which will aid in generating methods for interacting with the system, debugging errors, and keeping track of one's place in the system. Conceptual training based on a system model will be superior to procedural training in the mechanics of the system. We performed a laboratory experiment with two training conditions (conceptual and procedural), with each condition split by gender. Forty-three subjects participated in the experiment, but only 28 were able to reach the minimum competency level required to complete the experiment. We captured complete time-stamped online protocols for all subjects. The data analysis incorporated the monitoring data, personal characteristics, affective variables, and interview data in which subjcts described how they thought the system worked (an articulation of the model). As predicted, the conceptual training had no effect on the ability to perform simple, procedural tasks, but subjects trained conceptually performed better on complex tasks which required extrapolation from the basic operations of the system. Subjects had difficulty articulating a model of the system, and we found no differences in articulation by condition. The few gender differences that were found were trivial. The high number of subjects who were unable to reach the minimum competency level (a one-third drop out rate) suggests that Boolean information retrieval is a difficult task. The performance differences suggest that conceptual training is superior to procedural training, given that the user will need to do more than the simplest tasks.