Evaluation of the quality of consumer health information web sites: Diabetes tool development and impact of sponsorship characteristics

Context. Most existing tools for measuring the quality of Internet health information focus almost exclusively on structural criteria or other proxies for quality information such as sponsorship characteristics rather than evaluating actual accuracy and comprehensiveness. This research sought to: develop a health Web site quality evaluation model; investigate the impact of Web site sponsorship on the quality of health information; and assess the relationship between sites' structural characteristics and their performance on a set of accuracy and comprehensiveness measures.; Methods. An objective, systematic model was developed to evaluate Internet diabetes information based on a quality-of-care measurement framework. The investigator developed an abstraction tool and trained an external reviewer on its use. The tool included 7 structural measures and 34 performance measures created by using evidence-based practice guidelines and experts' judgments of accuracy and comprehensiveness.; Results. It took an average of 30.26 minutes to review each of 90 Web sites. Substantial variation existed in all categories, with overall scores following a normal distribution and ranging from 15 to 95 percent, with a mean of 50 percent, a median of 51 percent, and an interquartile range of 27 percent. The correlation (Spearman rank) coefficient between structural criteria and performance measures was moderate (0.42). T-tests revealed no statistically significant impact in pairwise comparisons of sponsorship characteristics. Holding all other factors constant, only one independent variable was related to Web site quality (for-profit status had a negative influence; p value of 0.03), but this impact had very little explanatory power (the adjusted R 2 was only 0.02). Lin's concordance correlation coefficient to assess agreement between the two raters produced a rho of 0.761 and a Pearson's r of 0.769, suggesting moderate agreement. The average agreement between raters for the performance measures was 0.80.; Conclusion. Diabetes Web site quality varies widely. Sponsorship characteristics and other proxies for information quality are not highly related to accuracy and comprehensiveness. The development and implementation of a reliable and valid method for evaluating the quality of Internet health sites could provide lay people with a tool to identify useful content more easily and distinguish between beneficial and misleading information.