Revision And Evaluation Of COPD-PRO Scale Based On Disease-syndrome Combination Theory

ObjectiveTo develop and evaluate the modified chronic obstructive pulmonary disease patient-reported outcome scale(mCOPD-PRO)and its computerized adaptive testing version by modifying previous COPD-PRO.Then "general process of modifying the PRO instruments" was preliminarily proposed,providing ideas and methods for related researches in other diseases.MethodsStudy 1:The development of mCOPD-PROField survey technology was used for further study based on the preliminary mCOPD-PRO.The items were analyzed and selected by combining classical test theory methods(discrete trend method,correlation coefficient method,Cronbach's alpha coefficient method,factor analysis method,etc.)and item response theory methods(discrimination,difficulty,information of items,item characteristic curve and item information curve)to develop the mCOPD-PRO.Study 2:The assessment of mCOPD-PROThe psychometric properties of mCOPD-PRO were evaluated in terms of reliability,validity,responsiveness and feasibility based on the field survey data;the item attributes were evaluated by item response theory and differential item function analysis;the distribution characteristics of response options were evaluated by ceiling and floor effects.Study 3:The preliminary study on the computerized adaptive testing version of mCOPD-PRO The calibration parameters of item pools of mCOPD-PRO were determined using item response theory.The initial item,item selection rule,ability estimation method and stopping rule were further set based on Concerto platform to establish the computerized adaptive testing model and form the computerized adaptive testing version of mCOPD-PRO.Simulation test was conducted,and then the reliability and validity of the computerized adaptive testing version of mCOPD-PRO were preliminarily evaluated.Study 4:Preliminary proposal of "general process of modifying the PRO instruments""General process of modifying the PRO instruments" was preliminarily proposed according to the international criteria for scale development,the important achievements in the field of quality of life in China,as well as our mCOPD-PRO work.Results Study 1:The development of mCOPD-PRO1 General conditions271 questionnaires were issued and 267 were retrieved.Three questionnaires were considered invalid because the data missing was more than 20%,and the remaining 264 questionnaires were valid.2 The item analysis and selection based on classical test theoryThe discrete trend method,correlation coefficient method,factor analysis method,and Cronbach's alpha coefficient method were used;accordingly,nine,five,five and one item were removed,respectively.3 The item analysis and selection based on item response theoryOne-dimensional test supported the one-dimensional assumption.Eighteen items were removed by comprehensively analyzing discrimination parameter(a),difficulty parameter(b),item characteristic curve,and item information curve.4 The formation of mCOPD-PROTwo items were merged and thirteen items were removed according to the methods above combined with panel discussion.The item pool was further optimized,and the mCOPD-PRO was formed,which consisted of three domains,namely nine facets(twenty-seven items).The physiological domain included four facets(seventeen items),psychological domain two facets(seven items),and surrounding domain three facets(three items).Study 2:The assessment of mCOPD-PRO1 General conditions370 questionnaires were issued and 368 were retrieved.Two questionnaires were considered invalid because the data missing was more than 20%,and the remaining 366 questionnaires were valid.2 Psychometric properties of mCOPD-PRO2.1 ReliabilityThe Cronbach's alpha coefficient of mCOPD-PRO was 0.954,and the Cronbach's alpha coefficient of physiological domain,psychological domain and surrounding domain were 0.930,0.929 and 0.673,respectively.2.2 Validity2.2.1 Content validity The patients and/or experts participated in the links,including modifying the conceptual framework,establishing the item pool,selecting the items,and forming the mCOPD-PRO,ensuring that mCOPD-PRO had a good content validity.The correlation coefficients between the scores of each item from physiological domain and the domain scores ranged from 0.429 to 0.775;the correlation coefficients between the scores of each item from psychological domain and the domain scores ranged from 0.670 to 0.902;the correlation coefficients between the scores of each item from surrounding domain and the domain scores ranged from 0.709 to 0.808.2.2.2 Construct validity Confirmatory factor analysis showed that comparative fit index(CFI),incremental fit index(IFI),non-normed fit index(NNFI),standardized root mean square residual(SRMR)and root mean square error of approximate(RMSEA)were 0.91,0.91,0.90,0.11 and 0.16,respectively.The factor loadings of each item ranged from 0.43 to 0.97.2.2.3 Criterion validity The correlation coefficient between mCOPD-PRO total scores and COPD assessment test scores and Modified Medical Research Council dyspnea scale(mMRC)scores was 0.771 and 0.651,respectively;the correlation coefficient between physiological domain scores and COPD assessment test scores and mMRC scores was 0.792 and 0.676.respectively;the correlation coefficient between psychological domain scores and COPD assessment test scores and mMRC scores was 0.564 and 0.449,respectively;the correlation coefficient between surrounding domain scores and COPD assessment test scores and mMRC scores was 0.571 and 0.514,respectively2.3 ResponsivenessThe difference of mCOPD-PRO total scores and domain scores between mild/moderate group and severe/extremely severe group was statistically significant(all P<0.01).2.4 FeasibilityThe acceptance rate and completion rate of mCOPD-PRO were both 99.5%,and the average completion time was 8.4 minutes.3 Item analysis and evaluation based on item response theoryOne-dimensional test supported the one-dimensional assumption.The discrimination parameter(a)of all items ranged from 1.05 to 2.71.The difficulty parameter(b)ranged from-3.08 to 3.65 for all items,and from-3.0 to 3.0 for 24 items among them.The first and fifth item characteristic curve showed monotonous changes,and the second,third and fourth curve showed normal distribution except for individual items.The maximum value of total test information was 34.224.4 Differential item function analysisOrdinal Logistic regression analysis showed that the P value of the third item was less than 0.01 when the domain scores were set as matching variable;the P value of the third and ninth item were both less than 0.01 when mCOPD-PRO total scores were set as matching variable.On the whole,the model fit of regression equation is not good.5 Ceiling and floor effectsThe ratio of the highest response ranged from 2.7%to 12.3%,and the ratio of the lowest response ranged from 3.8%to 43.7%.Study 3:The preliminary study on the computerized adaptive testing version of mCOPD-PRO 1 Calibration of item parameters based on item response theoryThe discrimination parameter(a)of all items ranged from 1.05 to 2.71.The difficulty parameter(b)ranged from-3.08 to 3.65 for all items,and from-3.0 to 3.0 for 24 items among them.2 Establishment of computerized adaptive testing model for mCOPD-PROThe computerized adaptive testing model established based on Concerto platform included item bank module,algorithm module,test system module,score report module and management system module.3 Formation of computerized adaptive testing version of mCOPD-PROThe computerized adaptive testing version of mCOPD-PRO was formed.At least five links(Login,Run,Test,Feedback and End)were needed to complete the whole test.4 Simulation testWith different sample sizes of simulated data(60 cases,100 cases,300 cases,500 cases,1000 cases,3000 cases and 5000 cases),the items used were all relatively discrete,the average number of items used was all 10,the correlation coefficient between simulated theta and real theta ranged from 0.970 to 0.976,and the average standard error of measurement(SEM)ranged from 0.290 to 0.291 using Random method.The items used were all relatively centralized,the average number of items used was all 7,the correlation coefficient between simulated theta and real theta ranged from 0.968 to 0.979,and the average SEM ranged from 0.289 to 0.292 using Maximum Fisher Information(MFI)method.5 Preliminary evaluation of computerized adaptive testing version of mCOPD-PRO.5.1 ReliabilityThe measurement reliability(r)ranged from 0.910 to 0.922 using Random method;the measurement reliability(r)ranges from 0.910 to 0.924 using MFI method.The correlation coefficient of the measurement reliability(r)between the two methods was 0.267(P=0.040);there was no significant difference in the measurement reliability(r)between the two methods(P=0.637).5.2 Validity5.2.1 Content validity As was mentioned above,mCOPD-PRO had good content validity.The computerized adaptive testing of mCOPD-PRO was based on twenty-seven items from mCOPD-PRO.Therefore,it could be considered that the computerized adaptive testing version of mCOPD-PRO had good content validity.5.2.2 Criterion validity The correlation coefficient between theta and COPD assessment test scores and mMRC scores using Random method was 0.628 and 0.540(P<0.001;P<0.001),respectively;the correlation coefficient between theta and COPD assessment test scores and mMRC scores using MFI method was 0.347 and 0.328(P=0.007;P=0.010),respectively.6 Comparisons between computerized adaptive testing version and traditional paper-and-pencil version of mCOPD-PROThe initial item of traditional paper-and-pencil version of mCOPD-PRO was always "Did you have a cough?";the initial item of computerized adaptive testing was random using Random method,and was always "Was your chest tightness aggravated by daily activities?" using MFI method.Twenty-seven items need to be tested in the traditional paper-and-pencil version of mCOPD-PRO.About eleven items(reducing by 59.3%)on average need to be tested in the computerized adaptive testing version of mCOPD-PRO using Random method,at least eight items(four cases)and at most twenty-six items(one case);about seven items(reducing by 74.1%)on average need to be tested using MFI method,at least six items(thirty-six cases)and at most twenty items(one case).The correlation coefficients between theta in the computerized adaptive testing version using Random method and mCOPD-PRO total scores,physiological domain scores,psychological domain scores and surrounding domain scores in the traditional paper-and-pencil version ranged from 0.623 to 0.919(all P<0.001);the correlation coefficients between theta using MFI method and mCOPD-PRO total scores,physiological domain scores,psychological domain scores and surrounding domain scores in the traditional paper-and-pencil version ranged from 0.535 to 0.760(all P<0.001).Study 4:Preliminary proposal of "general process of modifying the PRO instruments""General process of modifying the PRO instruments" consisting of three stages(namely fourteen links)was preliminarily proposed for the first time.The first stage(namely preparation stage)include analyzing the necessity of modifying instrument,establishing the modification group,presetting the basic characteristics of the modified instrument,and standardizing the basic concepts and terminology;the second stage(namely implementation stage)included modifying the conceptual framework,establishing the item pool,reviewing the items,forming the draft for the modified instrument,pre-survey,analyzing and screening the items,and forming the modified instrument;the third stage(improvement stage)included field survey,evaluating the modified instrument,and forming the network version of the modified instrument.Conclusions1 mCOPD-PRO consists of 3 domains,namely nine facets(twenty-seven items),which is derived from COPD-PRO,and has some optimization.2 mCOPD-PRO has good reliability,validity,responsiveness and clinical feasibility.The items have good attributes.No items have ceiling effects,and some items have floor effects.Individual items may have differential item function.3 The computerized adaptive testing version of mCOPD-PRO has good reliability and validity,and can improve test efficiency.4 "General process of modifying the PRO instruments" consisting of three stages(namely fourteen links)was preliminarily proposed for the first time.5 The key technologies of modifying,developing,evaluating the PRO instruments as well as computerized adaptive testing can provide a basis for improving the efficacy evaluation index system for respiratory diseases,and can also provide a reference for other diseases.However,this study also has some limitations.Further studies are needed.