Assessing the quality of behavioral health care and health plans using consumer reports and ratings

Interest in comparing consumer assessments of quality across health plans has grown. The Experience of Care and Health Outcomes (ECHO(TM)) Survey, a CAHPSRTM instrument, facilitates the standardized collection of consumers' evaluations of their behavioral health care and the plan that manages that care. The usefulness and fairness of plan comparisons based on these assessments depends on the extent to which summary measures are reliable (Paper 1), plan differences in casemix are taken into account (Paper 2), and group differences reflect quality differences versus reporting biases (Paper 3).; Factor analysis of data from 4,068 enrollees in 21 plans suggested four composites for summarizing consumers' reports about 22 specific experiences. Similar factor structures appeared for commercial and Medicaid respondents. Compared to the factor-based measures, four composites defined a priori by combining conceptually related items with the same response scale exhibited better or similar internal consistency and plan-level reliability. The a priori composites Timely Access to Care, Patient-Provider Interaction, Treatment Information, and Plan Approval and Service were recommended for reporting along with overall ratings.; A casemix adjustment model was developed by identifying consumer characteristics that were strongly related to assessments and varied significantly among plans. Mental health, general health, alcohol/drug use, age, education, and race/ethnicity were selected. Income was important in the commercial sample, and gender was important in the Medicaid sample. The impact of adjustment on plan scores was modest, but substantial enough to change plan rankings.; Group differences in reports and ratings may arise because certain types of consumers tend to experience better or worse quality of care or give more or less favorable evaluations compared to others. Multivariate linear regression was used to analyze the relationships between consumer characteristics, reports, and ratings. Adjusting for reports about experiences reduced the effects of health status, and to a smaller degree education and age, on rating scores. If reports are less subjective than ratings, these findings suggest group differences in ratings may reflect differences in experiences more than reporting biases. Plan-by-health-status interactions were significant in models predicting ratings and reports, indicating differences between consumers in better and worse health were greater in some plans than others.