| Diagnostic decision support systems have a long tradition in medicine, but have not typically been integrated with clinical information systems. I describe five studies that were performed during the development, implementation, and evaluation efforts for a real time diagnostic decision support system. The system's objective was to automatically identify patients likely to have pneumonia. The real time system used only data routinely collected in the computerized patient record during a patient's encounter in the emergency department. The automatic identification of pneumonia patients was used to initiate the computerized calculation of a pneumonia risk assessment instrument.; The first study describes the development of the diagnostic system using an emergency department data set from the HELP clinical information system at LDS Hospital, Salt Lake City, Utah. The second study describes the implementation of the system and illustrates the operational functions. The third study examined whether data from the HELP System could be used for the computerized evaluation of the pneumonia risk assessment instrument. This study showed that computerized evaluation generated an accurate risk class for 86 percent of hospitalized pneumonia patients. The fourth study reports the design and planning of the system's prospective evaluation in a clinical environment. The paper addresses important issues that influenced the study design, such as verification bias and disease prevalence. Different study designs were discussed with respect to the targeted users and the clinical setting, and a feasible approach for the creation of a valid gold standard diagnosis for pneumonia was proposed. The final study describes the system's prospective clinical evaluation. During a 5-month study period, the system computed a probability of pneumonia in real time for 10,828 patients, of whom 265 patients had pneumonia. The diagnostic accuracy, determined by the area under the receiver operating characteristic curve, was 0.942.; In summary, this project describes the development, implementation, and evaluation of a fully automatic, real time, diagnostic system that is integrated with a clinical information system. The system can be used to initiate pneumonia specific risk assessment tools and guidelines and to support the computerized guideline implementation for delivering recommendations at the point of patient care. |
