| After fifteen years of research most medical expert systems are not used by physicians. I decided to find out why this is so and what can be done to make these systems more attractive and effective, by studying reactions of medical personnel to the clinical use of an expert system for stroke. The major problem is that the physicians and computer scientists developing these systems have done little research in measuring the needs and attitudes of users. This thesis describes a methodology for evaluating medical expert system interfaces that combines methods from anthropology, psychology, sociology, and computer science. This methodology was developed as I carried out the evaluation of the Stroke Consultant at Michael Reese Hospital, a large teaching hospital in Chicago. The Stroke Consultant is a medical expert system developed to assist physicians in the diagnosis and management of stroke.; There were three main themes addressed in this research: (1) evaluation techniques, (2) application of these techniques in a natural setting, and (3) evaluation of the human-computer interface of the Stroke Consultant. The major techniques for the collection of data were embedded evaluation techniques, surveys and interviews with medical personnel, and participant observation.; Embedded evaluation techniques include Capture/Playback and Performance Analysis. The Capture/Playback tool allows for the 'capturing' of sessions which can later be reviewed. This tool also maintains statistics on which features of the system are being used, user errors, requests for on-line help, and system usage. The Performance Analysis tool measures the user response time, system response time, and the system display rate.; The results of this research suggest that the system has value but that there are many ways to make it more attractive and more convenient to use. It was clear that health care personnel will not use a system that just recommends a diagnosis. They will demand more information, such as definitions, references, explanations, reports, and database capabilities. The user's predisposition, their experiences with the computer system, and the social environment of the hospital all had an impact on their response to the Stroke Consultant. |
