Validation of an instrument to assess Web-based employee compliance training materials in academic medical centers as reviewed by institutional experts in compliance content, information technology, and instructional design

he current regulatory environment related to employee compliance training (ECT) in an Academic Medical Center (AMC) requires accountability and practicability. The need for efficient, quality ECT is critical, but standards are either not empirically based or applicable. Therefore, the purpose of this study was to measure the validity, reliability, and overall effectiveness of an instrument for the assessment of Web-based ECT materials in an AMC.;A panel of nine expert raters in compliance content, information technology, and instructional design was asked to assess three Web-based ECT modules on blood borne pathogens with an evaluation instrument developed by the Multimedia Educational Resource for Learning and Teaching On-line (MERLOT) organization. The instrument focused on three dimensions: quality of content (QoC), potential effectiveness as a teaching/learning tool (Effect), and ease of use (EoU).;The instrument was then converted to a Web-based questionnaire via Survey TrackerRTM. Subsequently, 130 members of the Group on Information Resources of the American Association of Medical Colleges with access to Email and the Internet, who were not in senior administration positions, were electronically requested to access and review an on-line BBP training module, use the on-line instrument to assess QoC, Effect, and EoU, evaluate level of satisfaction with the instrument (OEI), and identify their area of expertise as content (C), instructional design (ID), or information technology (IT).;The raters were in agreement (within 1) within their areas of expertise. All correlation values among raters were =/>7.00, achieving statistically or practical significance, except ID raters...