Can we ask better questions? An examination of the respondent -generated interval protocol

We describe a factorial experiment in the use of the RGI protocol involving recall of answers to factual questions. The RGI protocol involves asking sample survey respondents for a usage quantity, and a lower and upper value that bound their answer. The experiment is embedded within a sample survey by an HMO to investigate the quality of health care. We ask the same substantive questions of different groups of survey respondents to compare the results regarding accuracy and response rate.;Additionally, we performed computer simulation experiments to examine the behavior of the traditional sample mean and confidence intervals against the posterior mean and credibility intervals generated by Bayesian hierarchical modeling.;We have found the traditional methods are not the best in our pilot study in statistics classrooms. We have found better estimation for complex questions by using alternative methods like the quartile estimator or Bayesian posterior means. We also have found alternative interval estimators with a higher coverage percentage than the usual confidence interval.;In the HMO study, the sample mean was the best point estimator, however, using salience questions can improve the accuracy. We have found that the item response rate is higher for traditional methods than the RGI protocol. Additionally, respondents make their choices of which to answer due to perceived accuracy and ease.;The computer simulations have shown that the sample mean is not nearly as accurate as the posterior mean. We have also analyzed various methods of assessing the variance parameter tau. The average method utilizing the average of the lower bounds and the average of the upper bounds is best for estimating the posterior mean, but has poor coverage in its credibility interval. The range method utilizing the minimum of the lower bounds and the maximum of the upper bounds is best for covering the true mean in computer simulations.;The results we have seen depend on the complexity of the question and the sequence of the options presented to the respondent. Asking for additional information, either bounds or a salience question, can improve the accuracy of the response over traditional methods.