| Objective:To investigate the methodological issues of acupuncture clinical trials in intervention,control selection and statistical analysis.To investigate the relationship between acupuncture characteristics and therapeutic effects,and explore the factors associated with standardized report of acupuncture treatment.To explore whether sham acupuncture has specific therapeutic effects.To investigate the methodological quality in different journals,and evaluate the robustness of results.To conduct a randomized controlled trial to validate and construct a guideline for acupuncture clinical trials to provide a reference for promoting high-quality clinical evidence.To systematically review the application of machine learning in acupuncture research and explore the possibility of individualized treatment effect prediction for acupuncture clinical trials.Methods:Pub Med database was searched from January 2010 to December 2019 for acupuncture clinical trials,limited to core clinical journals and complementary alternative medicine journals.Four investigators independently and in duplicate screened the literature using pre-specified eligibility criteria,and extracted information regarding study characteristics,acupuncture interventions,control selection and statistical analysis.1)Survey of acupuncture interventions.(i)Descriptive statistics was used to summarize the patterns of acupoint selection for knee osteoarthritis,migraine,primary dysmenorrhea,and hypertension.(ii)Based on the STRICTA,19 items were constructed to evaluate the standardized report on the details of acupuncture intervention.Univariate and multivariate regression models were conducted,with seven prespecified factors,to explore the factors associated with the standardized report of acupuncture interventions.(iii)Taking chronic pain disorders as the study carrier and pain improvement as the outcome,univariate meta-regression analysis was used to explore the effects of acupuncture intervention characteristics(i.e.,type of acupuncture,acupoint prescription,response sought,deqi,needle stimulation,retention time,needle diameter,total number of treatment sessions,frequency of treatment sessions).2)Survey of control selection.(i)Taking chronic pain disorders as the study carrier,and a network meta-analysis was used to investigate whether there were differences between sham/placebo acupuncture and waiting-list/blank controls.If no results that direct compared sham/placebo acupuncture versus waiting-list/blank controls were available in the included studies,the adjusted indirect comparison method was used;if results that both direct and indirect comparison were available,the mixed treatment effect method was used,and the direct and indirect comparison evidence were combined by means of inverse variance.(ii)The treatment effects of each group were expressed as standardized mean differences(SMD)(Hedge’g correction).The ratio of the natural outcome effect of the disease to overall effect of true acupuncture group was used to calculate the proportion of the natural outcome effect in the true acupuncture group and its 95% CI.The ratio of the overall effect of sham acupuncture to the overall effect of acupuncture intervention was used to calculate the proportion of contextual effects(placebo effect and natural disease outcome effect)in acupuncture intervention and its 95% CI.One-arm meta-analysis was used to calculate the proportion and 95% CI to explore the placebo effect of sham acupuncture.If the proportions did not meet the normal distribution,data transformation could be conducted to make it meet or close to normal distribution.All meta-analyses were performed using a random effects model.3)Survey of statistical analysis method.(i)Methodological quality of the sample size estimation,primary outcome,analysis of repeated measures data,covariate adjustment,lost to follow-up information,subgroup analysis,and sensitivity analysis were evaluated separately with pre-specified items,and the methodological quality was compared across journals using Mann-Whitney U test.One-arm meta-analysis was used to combine the rates of lost to follow-up for individual studies,with a random-effects model.If the rates did not meet the normal distribution,data transformation could be conducted to make it meet or close to normal distribution.Subgroup analysis was performed by different journals(core clinical journals vs.complementary alternative medicine journals).(ii)The Fragility Index(FI)and Continuous Fragility Index(CFI)were used to evaluate the robustness of the results for dichotomous outcomes and continuous variables.The Spearman correlation was used to analyze the relationship between the study characteristics(sample size,followup time,and year of publication)and the FI as well as the CFI.Based on above methodological studies,a large sample(n=666),multicenter randomized controlled trial of acupuncture for knee osteoarthritis was conducted to test the above key methodology.Verification of the application of acupuncture clinical trials in standardization of key methods of acupuncture intervention,control selection,and statistical analysis.a)A standardized acupuncture intervention protocol was developed,such as identification,measurement and selection of acupoints,direction of needle insertion,acupuncture techniques.b)Selection of control was based on research purpose.c)A statistical analysis plan was prespecified,with appropriate analysis methods.The primary outcome was the change of Western Ontario and Mc Master Universities Osteoarthritis Index(WOMAC)total score from baseline to 16 weeks.The change from baseline in WOMAC total score over time was analyzed by fitting a linear mixed-effect model using baseline value as covariate,treatment,visit,and interaction between treatment and visit as the fixed effects,sites and individuals as the random effects.Four prespecified subgroup analyses of Kellgren-Lawrence criteria,BMI,duration of disease,and type of KOA were conducted for the primary outcome.A sensitivity analysis was conducted for the primary outcome basing on the per-protocol population.Based on the above methodology and empirical research,the guiding principles for acupuncture clinical trials were formulated,and the final principles were constructed through expert panel discussions and expert consensus meetings.Exploration of individualized prediction methods.Pub Med and CNKI databases were searched for acupuncture studies that use machine learning for data mining,classification or prediction from inception to May 2021.Descriptive statistics was used to summarize the application of machine learning in the acupuncture research and the machine learning algorithms that can be used to predict individualize effects.Based on clinical data,a patient cohort was formed,Bayes Bayes,support vector machine(SVM＿linear and SVM＿RBF),random forests(RF),Ada Boost,neural network(MLP and CNN)algorithms were used to constructed prediction models,then the best model were selected to predict the therapeutic effect among acupoints.Results:A total of 319 randomized controlled trials were included,of which,51 were published in core clinical journals and 268 in complementary alternative medicine journals.1)Survey of acupuncture interventions.(i)Acupoint prescriptions for knee osteoarthritis,migraine,primary dysmenorrhea,and hypertension were mostly following the meridians,supplemented by evidence-based and localized acupoints,and the selected acupoints corresponded to their etiology and pathogenesis.(ii)Acupuncture clinical trials had serious information deficits in the number of needles used(21.9%),direction/angle of insertions(31.3%),patient posture(32%),information on disinfection of acupoints(28.2%),and details of adjunctive measures(22.9%).The median standardized reporting score for acupuncture interventions was13 with an interquartile range of 11-14(core clinical journals: median score 13 with an interquartile range of 12-15;complementary alternative medicine journals: median score 13 with an interquartile range of 11-14).The overall intervention standardization of core clinical journals is better than that of complementary alternative medicine journals(p=0.016).Univariate regression analysis showed that core clinical journals(β=1.00,p=0.02),China(β=0.81,p=0.01),undergoing registration(β=1.07,p<0.001),reporting protocol(β=2.09,p<0.001),funding(β=1.00,p=0.002),and more than 6authors(β=1.10 p < 0.001)of clinical trials had more standardized reporting among acupuncture interventions.Multiple linear regression analysis found that only trials in China(β=0.63,p=0.05)and reporting study protocols(β=1.46,p=0.02)were associated with standardized reporting quality among acupuncture interventions.(iii)In the trials that compared acupuncture versus sham needling/blank control,univariate meta-regression analyses showed that there was no evidence that the characteristics of acupuncture intervention would change the effect of acupuncture on pain outcomes(p >0.05).In the trials that compared acupuncture versus other positive control,univariate meta-regression analyses showed that: type of acupuncture(p=0.02),individualized acupoint prescription(p=0.02)and acupuncturist qualification(p=0.03)would change the effect of acupuncture;other characteristics were not significant for acupuncture effect(p>0.05).2)Survey of control selection.Controls were mostly sham/placebo acupuncture(41.4%)and waiting-list(32.9%),and the most frequently was sham acupoints,followed by blunt needle device,and shallow needling.(i)Network meta-analysis showed no statistical difference between sham/placebo acupuncture and waitinglist/blank control in patient with chronic pain(SMD =-0.15,95% CI [-0.91,0.62]).(ii)The placebo effect accounted for a mean of 32%(0.32,95% CI [0.02,0.62])of the overall treatment effect in true acupuncture group and 68% could be interpreted as contextual effects(placebo effect and natural disease outcome effect)to acupuncture.The overall effect in the sham acupuncture group compared with the overall effect in the acupuncture group averaged 52%(0.52,95% CI [0.18,0.87]),and the 48% overall effect of the acupuncture group could be explained as acupuncture-specific therapeutic effects,20% of the overall effect can be explained as a placebo effect(or including the specific effects of sham acupuncture).3)Survey of statistical analysis method.(i)There were several issues in the statistical analysis,including failure to conduct sample size estimation(53.9%),prespecify primary outcomes(51.1%),analyze repeated measures using appropriate statistical analysis methods(57.9%),and address possible central effects in multicenter trials(77.8%)and so on.The overall lost to follow-up rate was 9%(95% CI [0.07,0.11]),and subgroup results found no difference in lost to follow-up rates across journal trials(interaction p=0.54).Better methodological quality in the sample size estimation,primary outcome,repeated measures information,and lost to follow-up information in core clinical journals than in complementary alternative medicine journals(p<0.001).(ii)The median FI was 4,with an interquartile range of 1-13.FI was positively correlated with sample size(Spearman r=0.68,95% CI [0.39,0.85],p=0.0002),duration of follow-up(Spearman r=0.59,95% CI [0.26,0.80],p=0.002).The median CFI was 15,with an interquartile range of 8-22.CFI was positively correlated with sample size(Spearman r=0.75,95% CI [0.50,0.88],p<0.001).Randomized controlled trial of acupuncture treatment of knee osteoarthritis(empirical research).A total of 666 patients were included and 625(93.8%)completed the trial.The patients of lost to follow-up were balanced among the groups.(i)This clinical trial strictly followed the standardized acupuncture intervention protocol and objectively quantified the details of acupuncture intervention;in terms of control selection,a control was established based on the research purpose,and the control selection was made according to the pressure-pain threshold ranking of candidate acupoints/tender points(that is,five points with highest pressure-pain threshold were identified as low-sensitized points and served as a control group for acupuncture intervention),and a waiting treatment group was set up to eliminate the influence of the patient’s own disease relief;a statistical analysis plan was set in advance and an appropriate analysis method was selected.(ii)In the intention-to-treat analysis,participants in the high-sensitization group versus low-sensitization group did not result in statistically significant difference in reducing WOMAC total score(adjusted MD=2.21,95% CI [-2.51,6.92])at 16 weeks,but the high-sensitization group and lowsensitization group significantly reduced WOMAC total score compared with waitinglist group(adjusted MD=-9.77,95% CI [-14.47,-5.07],CFI=39;adjusted MD=-11.97,95% CI [-16.71,-7.24],CFI=48).Similar results were observed in the per-protocol analysis.Subgroup analyses by Kellgren-Lawrence criteria(p=0.93),BMI(p=0.79),duration of disease(p=0.33),type of KOA(p=0.82),and treatment at 16 weeks showed no significant differences between high-sensitization and low-sensitization groups.Construction of clinical trial guideline for acupuncture.The guideline involved design,implementation,and reporting,including 23 items.In addition to the general recommendations for randomized controlled trials,this guideline focus on the acupuncture intervention protocol,control selection,and statistical analysis.Exploration of individualized prediction methods.A total of 57 acupuncture studies involving machine learning algorithms were included.Among the included studies,49.1% aimed to use machine learning algorithms to solve pattern classification problems or predict the effects of acupuncture.The machine learning algorithms include SVM,ANN,RF,DT,Bayes and other supervised learning algorithms.SVM(SVM＿linear and SVM＿RBF),ANN(MLP and CNN),RF,Bayes may be better algorithms for individualized acupuncture prediction.A total of 444 patients were included.The AUC of the prediction model constructed by Bayes,SVM＿linear,SVM＿RBF,RF,Ada Boost,MLP and CNN algorithms is 0.38,0.52,0.43,0.40,0.47,0.52 and 0.56 respectively.CNN algorithm selected to predict the best combination of acupuncture points for patients,verifying that machine learning can be used to predict individualized effects of acupuncture.Conclusion:The reporting of intervention details in acupuncture clinical trials is generally standardized,with serious information missing in the number of needles used,direction/angle of insertions,patient posture,information on disinfection of acupoints,and details of adjunctive measures.Clinical trials in China and those with study protocols may be more standardized,and rigorous in terms of acupuncture interventions.The type of acupuncture,the type of acupoint prescription,and the acupuncturist with acupuncture practice may influence the efficacy of acupuncture.Most controls for acupuncture clinical trials are sham/placebo acupuncture or waiting-list/blank controls,and the most frequently is sham acupoints,followed by blunt needle device,and shallow needling.No statistical difference between sham/placebo acupuncture and waiting-list/blank controls is found,indicating no significant specific efficacy for sham acupuncture.In the true acupuncture group,about 48% of the effects can be attributed to acupuncture intervention,and 20% of the overall effect can be explained as a placebo effect(or including the specific effects of sham acupuncture),indicating that the specific effect of sham acupuncture is smaller.Due to the small sample size included in the study,further study should be used to validate this finding.There are several issues in the statistical analysis for acupuncture clinical trials.Compared with core clinical journals,trials in complementary alternative medicine journals need to be improved in terms of statistical analysis.The results of continuous variable results are relatively stable,but the dichotomous results may lack sufficient robustness.This finding is not fully representative of the overall study because the analysis is only conducted on the primary outcomes.Standardized acupuncture intervention,control selection,and statistical analysis can help reduce potential bias in research and improve the quality of evidence in acupuncture clinical trial.The guiding principles for acupuncture clinical trials have been established,which can provide methodological guidance for future such research.Based on acupuncture clinical trial data,machine learning algorithms used to achieve individualized treatment effect prediction are not ideal. |
PDF Full Text Request