Embedded tools for user guided evolutionary development of clinical information systems

Context. Clinicians use medical informatics systems to make their work easier and more effective. Yet such systems remain rare, in part because of difficulty communicating different professional paradigms between clinicians and software developers. Evaluation tools in other fields have enhanced such communication among system stakeholders.;Objective. Study feasibility and effectiveness of embedding evaluation tools into clinician workstations.;Design. Case study hypothesis-generating experiment using grounded theory qualitative analysis.;Setting. CWS Clinician Workstation project at University of Minnesota Hospitals and Clinics blood bank and four inpatient units.;Participants. Physicians and nurses using CWS and developers of CWS.;Intervention. Embedded evaluation tools for observation of stakeholder and system behavior and stakeholder surveys. Electronic mail for observation collection and automated reporting. Pre-addressed e-mail feedback for direct communication from clinicians to developers. Multi-layered security supporting personal e-mail for two-way communication among stakeholders. E-mail surveys allowing stakeholders to distribute questionnaires and collect responses.;Main outcome measures. Deployability of embedded evaluation tools. Tool use by stakeholders: quantity and content of feedback and e-mail messages from clinicians; observations selected and used by developers; survey questionnaires created by stakeholders, response rates, and result use.;Results. Embedded evaluation tools were developed. Unexpected resistance to clinician e-mail came from nursing administration, centered on perceived lack of resources for training, although CWS itself was deployed without training. Therefore, clinician e-mail and survey tools were not deployed. Most common clinician feedback messages were requests for features and display formats. Initial feedback averaged 25 messages per month, tapering irregularly by half over one year, with increases early each month. Developers perceived themselves understaffed for CWS, with few resources for feedback follow-up. User authentication was added to feedback access after several clinicians denied messages sent in their name. Developers used observations to track clinician usage, software behavior, and database response times.;Conclusions. Embedded evaluation tools can be successfully deployed in clinician workstations, help expose divergent concerns between stakeholder groups, and generate hypotheses. Is nursing training for e-mail a cultural difference from physicians, or a proxy for some other issue? Would volume and quality of clinician feedback increase over time with prompt developer follow-up?...