Developing Risk Prognostication Model To Reduce The Clinical Uncertainty Of Upper Respiratory Tract Infections And Evaluating Its Impacts On Promoting The Rational Use Of Antibiotics In Primary Care Facilities In China

BackgroundAt present,promoting the rational use of antibiotics and combating antibiotic resistance is the consensus in global.The irrational use of antibiotics in primary care facilities is a common challenge for all countries to tackle antibiotic resistance.Primary care facilities,especially village clinics in rural areas,are the hardest hit by irrational use of antibiotics in China.Upper respiratory tract infections(URTIs)are among one of the common diseases and the manifestation of irrational use of antibiotics.The main reason is that under the reality of economic development and infrastructure,village clinics generally lack of necessary examination equipment,which makes it difficult for village doctors to accurately judge whether the infection is caused by virus or bacterial infection only by self-reported clinical symptoms from patients such as cough,headache and fever.Therefore,in order to avoid the risk of complications or hospitalization of patients,village doctors usually adopt the risk-avoidance strategy of prescribing antibiotics "just in case",which antibiotics use is to defensive reduce the risk of hospitalization of patients with URTIs.It is common for village doctors to believe that it is professionally unacceptable for patients to get worse if they do not use antibiotics,and they will also come under pressure from patients,medical institutions,and peer pressure.Interventions have been conducted to regulate doctors’prescribing behavior,mainly by reforming the incentive mechanism of medical institutions,education,prescription review and regular disclosure of the results.These measures have been proven to be effective in reducing the use of antibiotics,but the sustainability and popularization of such studies are limited.A few studies have explored reducing antibiotic use by reducing clinical uncertainty.The hypothesis of this study:if village doctors can effectively distinguish between the high and low risk of future hospitalization of patients with URTIs,thereby reducing clinical uncertainty,it will significantly reduce the unnecessary use of antibiotics in low-risk patients and promote the rational use of antibiotics in patients with URTIs.The specific question is:how to reduce the clinical uncertainty of patients with URTIs through scientific methods,to reduce the defensive use of antibiotics by village doctors?ObjectivesThe aim of the present study is to use medical big data to develop a prediction model to provide clinical decision support for village doctors for decreasing the clinical uncertainty of URTIs,and reduce unnecessary antibiotic use in rural village clinics.A cluster-randomized controlled trial was carried out in the village clinic to verify the effect of the risk prediction model in reducing the clinical uncertainty of patients with URTIs and reducing the antibiotic prescription rate.Finally,we proposed a new strategy for rational use of antibiotics in line with the actual situation in primary care facilities in China,and achieved the goal of promoting the rational use of antibiotics and slowing down the emergence of antibiotic resistance.Research methodsThis project is funded by the National Natural Science Foundation of China in 2021,"Study on the construction of risk prediction model to reduce clinical uncertainty of respiratory tract infection and the evaluation of promoting the rational use of antibiotics in primary care"(grant number:72174109).The research content is mainly divided into three stages,and the methods used in each stage are as follows:In the first stage,the risk factors of clinical uncertainty in URTIs were identified and the index system was constructed.Literature analysis,key informant interview and Delphi expert consultation were used to collect data.Such qualitative data were analyzed using the methods of literature review,content analysis and Colaizzi analysis.In the second stage,based on the risk factor index system constructed in the first stage,the whole process medical data of 13.73 million patients with URTIs obtained from Shandong Guoshuai Health Big Data Center is used.Three machine learning algorithms,including support vector machine(SVM),random forest(RF)and extreme gradient boosting(XGBoost),were used to construct risk prediction models to reduce the clinical uncertainty of URTIs.DeLong test,AUC,F1 score,Kappa coefficient and other indicators were used to evaluate the accuracy,stability and consistency of the model,and the optimal model was selected.Then,a clinical decision support system(CDSS)based on the optimal risk prediction model was developed.In the third phase,a cluster-randomized controlled trial was conducted to evaluate the effect of CDSS based interventions on promoting rational use of antibiotics by village doctors.Village doctors from 28 village clinics were randomly divided into the intervention group and the control group,14 in each group.First,descriptive analysis,T-test,difference-in-differences(DID)and other methods were used to analyze the prescription data before and after the intervention to evaluate the effect of the interventions in promoting rational use of antibiotics.Secondly,the robustness was tested by transforming the time interval of the sample data,the fixed effect model,and the random effect model.Finally,key informant interviews and focus group discussions were used to evaluate the feasibility of implementing the interventions in primary care facilities and the acceptance of the interventions by village doctors.Results(1)Based on literature review,key informant interview and expert consultation,the risk factors were concentrated into four dimensions of individual characteristics,past medical history,clinical symptoms and physical examination,and an index system consisting of 42 risk factors was formed.This index system was used to construct a risk prediction model to reduce the clinical uncertainty of URTIs.(2)DeLong test showed that the difference between the ROC curve of the risk prediction model based on XGBoost and the SVM and RF risk prediction models was statistically significant(p<0.001),indicating that the prediction accuracy and sensitivity of the XGBoost model were significantly better than those of the other two models.The prediction accuracy of XGBoost model was 0.80,the sensitivity was 0.77,the specificity was 0.84,and the balance accuracy was 0.80.At the same time,the Kappa coefficient of the XGBoost model was also better than that of the other two models,which was 0.63(>0.6),indicating that the model prediction results were in good consistent with the actual classification results.Therefore,after the risk model was constructed,CDSS with the XGBoost model as the core was developed.(3)The DID results of the cluster-randomized controlled trial showed that the interventions based on the application of CDSS had significantly promoted the rational use of antibiotics by village doctors.The interventions had significantly reduced the antibiotic prescription rate by 14.3 percentage points(p<0.05),reducing the antibiotic prescription rate to 32.5%;At the same time,the interventions resulted in a significant reduction of approximately 4.4 percentage points in the rate of antibiotic co-prescribing(p<0.001).(4)After the intervention,the structure of antibiotic use in the intervention group was optimized:the antibiotics were mainly Access category,accounting for 78.2%;Watch category usage accounted for a decline,from 39.7%to 21.8%.In addition,the amount of traditional Chinese medicine prescriptions in the intervention group increased significantly(p<0.001).(5)Through qualitative interviews with village doctors,it was found that the intervention effectively reduced the clinical uncertainty of patients with URTIs.It reduced the unnecessary antibiotic prescribing behavior by reducing doctors’ concern about the risk of hospitalization for patients with URTIs.CDSS plays a timely and effective role in guiding village doctors in the process of medical service.Therefore,CDSS not only regulates the prescribing behavior of village doctors,but also enhances their awareness of rational use of antibiotics.The required conditions of CDSS matched the level of infrastructure at the primary care facilities,and with enough feasibility.Conclusions(1)The combination of machine learning methods and big data has significantly improved the accuracy of risk prediction model and ensured the effectiveness of CDSS.Medical big data provided a rich data basis,which effectively improved the problem of limited extrapolation of research conclusions due to small sample size.(2)Reducing the clinical uncertainty of patients with URTIs can significantly promote the rational use of antibiotics by village doctors.The use of CDSS by village doctors to predict the hospitalization risk of patients with URTIs without antibiotic use can effectively reduce the concern about the hospitalization risk of patients’ prognosis,thereby reducing the clinical uncertainty of patients in the process of diagnosis and treatment,and achieving the goal of effectively reducing the defensive antibiotic prescribing behavior of URTIs.Thus,it supports the hypothesis that helping primary care providers to effectively identify patients with URTIs who are at low or high risk of future hospitalization would significantly reduce unnecessary use of antibiotics.At the same time,this further answer the question on this topic.After the intervention,the antibiotic prescription rate decreased significantly,while the prescription rate of Chinese traditional medicine increased significantly,and showed a substitution effect on antibiotics.(3)The application of CDSS realizes the interventions in the process of clinical diagnosis and treatment,and it can continuously promote the rational use of antibiotics by village doctors.On the one hand,it can timely provide auxiliary guidance for village doctors during the clinical diagnosis and treatment;On the other hand,it can also standardize the medication behavior of village doctors through daily reminders,enhance the awareness of rational use of antibiotics,and make the promotion of rational use of antibiotics normally.(4)The interventions based on CDSS were adapted to the actual conditions in rural areas,and effectively relieve the problems of poor promotion and persistence of previous interventions such as health education and doctor’s prescription review.At present,the basic configuration of computers can be compatible with CDSS,which greatly improves the popularization of interventions in primary village clinics.At the same time,the computer operation ability accumulated by village doctors for basic public health service has laid a good manpower foundation for the long-term promotion and application of CDSS.Policy recommendations(1)Under the background of "focus on rural areas" in the 20th National Congress of the People’s Republic of China and "focus on rural areas and primary care" in the "Healthy China 2030 Plan Outline" governments should focus on primary care,strengthen the capacity building of antibiotic governance in primary care facilities,and effectively promote the rational use of antibiotics in primary care by formulating interventions or policies to reduce the clinical uncertainty of patients with URTIs.(2)Providing auxiliary information to primary care doctors during the diagnosis and treatment to reduce the clinical uncertainty of patients,and achieve the goal of promoting the sustainable and regular development of rational use of antibiotics in primary care facilities.Two key principles should be noted:Firstly,based on the reality that the basic construction level of primary care facilities is backward and the medical service capacity is insufficient,the form of auxiliary guidance should be adapted to the reality of rural areas;Secondly,continue to strengthen the construction of primary care information technology to help the scientific management(use)of antibiotics.(3)Formulate systematic policies to give full play to the advantages of "Traditional Chinese Medicine and western medicine",scientifically improve the substitution role of Traditional Chinese Medicine on antibiotics,and further standardize the development of Traditional Chinese Medicine,including pricing and sale.(4)Comply with the development of time,and make full use of new products such as big data and artificial intelligence to promote the rational use of antibiotics in primary care facilities.(5)Various forces should cooperate to promote the rational use of antibiotics through interdisciplinary cooperation.Strengths and limitationsInnovation points:(1)Based on the reality of backward basic laboratory equipment and insufficient medical service capacity in primary care facilities,the research integrates the identification of risk factors,big data analysis and application,and evaluation of interventions into a whole,and the results not only enrich the research data in this field,but also takes the essential problem leading to the irrational use of antibiotics as the starting point.It also provides a new perspective for the development and promotion of rational use of antibiotics in primary care facilities.(2)This study innovatively uses the method of "medical big data and artificial intelligence" to construct a risk prediction model.On the one hand,medical big data provides a rich data basis for this study.On the other hand,the combination of machine learning and medical big data has significantly improved the accuracy of model prediction.(3)The study is based on a cluster-randomized controlled trial,and uses the DID model to evaluate the effect of CDSS with risk prediction model as the core to assist village doctors in the rational use of antibiotics in rural areas,effectively controlling the grouping effect and time effect between the intervention group and the control group,and correctly screening the causal effect between the intervention measures and the rational use of antibiotics.At the same time,the CDSS is matched with the informatization construction level of the village clinic.By predicting the level of risk,it can continuously provide doctors with auxiliary information in the process of upper respiratory tract infections diagnosis and treatment,which not only realizes the "in-process"intervention of doctors in the process of clinical diagnosis and treatment,but also effectively improves the problems of poor continuity and promotion of traditional intervention.The CDSS with the risk prediction model as the core developed in this study matches the level of primary care facilities’ information construction and reality.By predicting the level of risk,it can continuously provide auxiliary information for doctors during diagnosis and treatment of URTIs,which not only realizes the interventions of village doctors in the process of clinical diagnosis and treatment,but also the interventions based on CDSS effectively relieve the problems of poor persistence and popularization of traditional intervention.Limitations:(1)This study conducted a six-month cluster-randomized controlled trial,and the effect of interventions on village doctors’ prescribing behavior may be delayed,and the intervention effect may be underestimated.We will conduct a longer-term intervention study to reveal the impact of the intervention on antibiotic prescribing behavior of primary care doctors.(2)The target of this project is the village doctors on the supplier side,aiming to improve the rational use of antibiotics.However,in the process of irrational use of antibiotics,the demand side,that is,the patients,will also play a role.Lack of demand-side studies will limit the effectiveness of interventions to promote rational use of antibiotics.In order to solve this problem,in the future research,it is necessary to establish a patient-centered research cohort to lay the foundation for the scientific implementation of research design and make the research conclusions more reliable and credible.