Establishment Of A Symptom Monitoring Index System Based On Patient-reported Outcomes

Background:Symptom management based on patient-reported outcome(PRO)is a major research interest for whole-course patient management in the coming future.Applying PRO-based symptoms as an index is a new concept to capture the disease burden of patients.Accurate symptom management provides a basis for determining the degree of disease severity,treatment efficacy,and quality of life of patients.Adenomyosis is a common benign gynecological disease,and pain-centered symptom clusters are the focus of managing patients with adenomyosis.Successful pain management not only reduces its severity,but also alleviates associated symptoms such as nausea,fatigue,and anxiety,thus significantly improving patient’s quality of life.Compared with the management of patients with malignant tumors,PRO in the management of benign diseases lags behind from the perspectives of methodology and application.Pain as the main clinical manifestations of treating adenomyosis,basic methodological and systematic management of symptoms are insufficient,whether in clinical practice,patient management,or clinical research.The lack of data hinders answering key questions to PRO applications(for example,Can the instrument’s content accurately reflect patients’ genuine feelings? Can instrument measurement performance keep consistent across patient populations? Are the applied parameters clinically significant?)Consequently,the measurement and statistical evidence supporting the reliability,validity,and interpretability of PRO are insufficient in the realworld,leaving PRO’s application in patient symptom monitoring and management still in its infancy.Purpose:Using a combination of artificial intelligence,psychometric,and statistics,this study aims to optimize and improve the methodological issues in the PRO symptom monitoring system.Therefore,PRO monitoring can identify patients’ symptoms and changes accurately and timely,allowing for early identification and treatment targeting at disease progression or adverse treatment reactions,and contributing to whole-course management of chronic diseases.1.Aims to explore whether specificity symptoms can be automatically identified completely and accurately,through doctor-patient communication data on We Chat and to determine whether NLP extract symptom items can replace part of expert interviews and manual identification in PRO instrument development.This exploration helps obtain a fuller picture of patients’ true feelings and needs,while reducing the burden on doctors,patients,and scientific research.A prerequisite for whole-course management in patients with adenomyosis is specific PRO monitoring tools.However,no specific symptom management tool is available yet to date.Establishing a new specificity PRO instument from scratch requires tremendous time and financial input.It deserves to consideration on how to leverage existing data to ensure not only the specificity and sensitivity of the items,but also the Validity and reliability of the symptom items from patients,with less money and time spent.The emergence of natural language processing(NLP)and machine learning approaches,which automatically extract and generate relevant patient-reported symptom information from free-text data sources through algorithms,can help overcome the challenges of manual fre-text analysis.We generated a draft version of symptom management scale of adenomyosis according to NLP and We Chat groupbased data.2.To make a comprehensive assessment of the AM-SAS,combined with Item Response Theory(IRT)and Classical Test Theory(CTT).The reliability,validity,and effectiveness of monitoring tools are quality assurance measures for the whole-course management of symptoms.CTT,as the most extensively applied psychological theory,defines the random error of patient measurement with reliability,validity,and other indicators.In fact,only observation scores can be obtained,rather than true scores,so a error scores are intermingled in the analysis.Resulted CTT can not describe the item characteristics.IRT,an emerging psychometric theory,is a representative of modern measurement theory,which establishs models with latent traits and item characteristic curves.Between the two test theories lies a complementary yet irreplaceable association.To assure the stability of the scale’s measurement performance in patients with heterogeneity,this study applies both CTT and IRT to validate AM-SAS,simultaneously.3.To explore statistical evidence that scores 0-10 should be classified as rank variables or continuous variables,and to determinate the optimal correlation threshold between the anchor and target symptoms.Evaluation of the application parameters is a prerequisite of the instrument application.As the FDA-recommended threshold for interpretatiing PRO clinical results,MCID has been widely applied.An anchor-based MCID formulation method is the most common one.Therefore,the criteria of anchor selection are the determinant of the clinical application.Based on practical experience and Cohen’s rules of thumb,experts recommend that at least moderate correlation between patient-reported outcome measures(PROM)and anchor,which was reported to be at least 0.3.However,some studies demand the correlation as high as 0.5,even recommendation on 0.7.These inconsistent statements only based on expert experience and discussion,lacking of quantitative evidence of the optimal correlation coefficient for anchor selection.We aimed to explore how correlation coefficient affects MCID,and to establish the optimal correlation thresholds between the anchor and PROM.Furthermore,it has not been determined whether the scores 0-10 should be used as a grade variable or a continuous variable.Method:1.After identified screenshots as text,through established custom dictionary,Synonyms lists,stop words,realized automatic segment word.Finally,we extract symptom keywords with term frequency-inverse document frequency(TF-IDF).A qualitative study was conducted in line with the US FDA’s development principles and procedures for the PRO instrument.Patients with adenomyosis were interviewed about their symptoms and functional interference in a semi-structured interview and quantitatively analyzed the interview text using grounded theory.We Compared the similarity and difference of extracting symptoms by different methods,to explore the feasibility of the doctor-patient communication text on We Chat in the development of PRO tools.Combining symptoms identified by two methods,we generated a draft version of symptom management scale of adenomyosis according to the expert discussion.2.The reliability,validity,and discrimination of the core symptoms of adenomyosis were validated using a combination of CTT,NLP,and qualitative interviews,to ensure the validity of the scale in practical application.At the same time,the appropriate IRT model was selected to validate the hypothesis of the model(unidimensionality and local independence),and test the authenticity of patients’ responses.The item fitting degree and discrimination degree were determined by the item characteristic curve and parameter estimation.3.Simulation data were based on the Copula function and a simulation database of anchor and symptom change scores was generated with different correlation conditions.The MCID of the pain(scores 0-10)was formulated,and the anchor was the changes in functional interference(-10 to 10 points).Simulation data were based on the Copula function and a simulation database of anchor and pain change scores was generated with different correlation conditions(from 0.1 to 0.9).Consider the 0-10 or-10 to 10 as ordinal variable,the baseline parameter,such as the frequency of each grade in anchor and change score of pain,was based on data from a cohort of patients with lung cancer surgery.Using Anova and Logistic regression model to compared the MCID under different coefficient in simulated data.Bootstrap resampling with 1,000 samples utilized to evaluate the stability of MCID for the optimal correlation.We defined stability as the largest proportion of threshold in 1,000 samples bootstrap.Under the correlation coefficient from0.1 to 0.9,the inflection point of the correlation coefficient was found according to the trajectory of the frequency of optimal cut point in the bootstrap resampling results.The obtained parameter selection results were employed in AM-SAS and MDAC-LC,a lung cancer scale,to calculate MCID.Results:1.Totally,9,074 screenshots of doctor-patient communication on We Chat were identified,with a total of 1,189,413 words.After natural language processing,contain 121,520 words from patient reports,we interviewed 29 patients with adenomyosis.Compared to qualitative interview,NLP recognized all the symptoms except for some privacy symptom in doctor-patient communication text.Issues relating to privacy,such as dyspareunia and breast tenderness were not mentioned in the We Chat group-based data.At the same time,surgery-related symptoms were extracted We Chat.For example,hypomenorrhea,Vaginal itching and Vaginal dryness,which are the postoperative recovery symptom or postoperative vaginitis and excluded from the list according to the expert opinion.2.AM-SAS demonstrated high internal consistency reliability,solid concurrent validity,and discriminant validity,according to the CTT results.In the development and external validation cohorts,results showed that AMSAS can distinguish between patients with pain interference,anemia,and relapse,significantly.Further testing the tool’s ability to distinguish patients with adenomyosis from those with other confounding conditions(e.g.,endometriosis and chocolate cysts)will be a valuable area for future research.The Generalized Partial Credit Model(GPCM)was selected for IRT,and the results verified the results of CTT.The patients’ s fitting statistics showed a high degree of fitting,proving that the response of this study was true.The GPCM model’s local independence test revealed that several items,particularly the three functional interference items had local dependencies abdominal pain,waist pain,and anus bearing-down pain.All items were highly discriminative,allowing patients with different abilities to be distinguished.Some items such as menorrhagia and waist pain presented greater difficulty in determining the severity of current patient symptoms,especially in 6 to 10.Except for the feeling of anus bearing-down pain,the fitting degree of items was acceptable.The total informativeness curve of the item dimension was between-2 and 0,indicating a moderate measurement accuracy at the individual level.3.The pain,anchor score,its standard deviation,and simulated correlation coefficient are closely to the pre-specified ones.The bootstrap results revealed that the stability of MCID gradually increased with the increase of correlation coefficient and the MCID was quite close between ANOVA and Logistic models,especially the results of the F value of ANOVA and the chi-square value of Logistic.The treatment of the scores 0-10 by quantitative or ordinal methods did not differ significantly in terms of statistical significance for calculating MCID.At the same time,a correlation of 0.3 between the anchor and the target symptom was found to be the minimum requirement(stability≥50%),and correlations at and above 0.5witnessed the stability of the MCID results maintain above 70%.Results demonstrated correlations at or above 0.5 as the optimal anchor-PROM correlation coefficient for the establishment of MCID.With the above criteria applied,the MCID value of lower abdominal pain in AM-SAS scored3.The MCID values for pain and fatigue in MDASI-LC were both 30% and scored 2.Conclusion:1.Based on NLP,We Chat group-based doctor-patient communication texts can catch most symptoms experienced from patients consulting,although items of privacy were missed due to the privacy of the mute-person communication group.In order to preserve the integrity and accuracy of the symptoms,and to avoid irrelevant symptom contained in the final version,we propose that after NLP,an expert consultation could be used to determine.2.AM-SAS can be utilized in clinical practice to assess the burden of symptoms in patients during the menstrual cycle.IRT can determine how distinct items are on a specific grading scale,which can be used for item optimization and screening.The PRO scale should be validated by integrating CTT and IRT.3.The 11-level scale of scores 0-10 can be used as a continuous variable to calculate MCID.Our study provides quantitative statistical evidence for anchor selection,in terms of the correlation between the PROM and the anchor.The empirical 0.3 threshold generated an acceptable stability of MCID,while 0.5 is preferable.Filled a gap of methodology pitfalls in establishing of MCID and reporting will better inform the application of MCID in clinical research and decision-making.