Establishment Of Indicators Of Process Quality Control Evaluation For Traditional Chinese Medicine Clinical Research

Traditional Chinese Medicine (TCM) clinical research must attach importance to the authenticity and reliability of the results, no matter what size, objective and content is. Therefore, conducting quality control of TCM clinical research to guarantee the authenticity and standardization, is particularly important and receives increasing concerns from the government and the public. Ministry of Science and Technology set up a "Scheme Optimization and Quality Control for TCM Clinical Research" project, in the "Prevention and Treatment of Difficult and Complicated Disease of TCM" plan of the "11th Five-year National Key Technology R&D Program" for the first time, to implement the project optimization and process quality management for TCM clinical research, through the establishment of four-level quality control and quality assurance system.Process quality control evaluation for TCM clinical research is to assess the quality of research activities during the whole process, focusing on goal attainment and implementation level. Based on quality monitoring results, we analyzed the quality management level in accordance with the scheduled target, and predicted the development trend. The final intention of quality control evaluation of TCM clinical research is to solve the problems, then adjust or correct the execution process in time.Objective:1. To establish indicators of process quality control evaluation for the first time, by enriching and optimizing the quantitative indicators of mid-term evaluation, balance the index number under different structural layers, extend range of assessment and realize quantitative evaluation of whole-process quality control standard of TCM clinical research ultimately.2. To evaluate the reliability and validity of indicators, and explore the application value in process quality control evaluation for TCM clinical research.Methods:First of all, we established a set of index system for quality control evaluation, targeting on key links in implementation process of TCM clinical research, then screened and optimized the indexes by means of expert investigation, ginger group discussions, subjective weighting, cluster analysis and correlation analysis, as well as feature analysis. Secondly the expert investigation method was used to determine the 14 remained core indicators’ subjective weighting, according to which the quality control evaluation score of a project was calculated. And the reliability and validity of the indicators were evaluated through structural equation model (SEM). Considering data’s skewed distribution, partial least squares (PLS) estimation method was used to calculate the objective weighting, according to which the score and the ranking of a project was computed. Finally to compare and analyze the two scores, we discussed the application of quantitative indicators in TCM clinical research.Results:1. Reliability analysis result of process quality control evaluation indicators following mid-term evaluation theoretical structural showed:The a coefficients of "research progress", "drugs management" and "sub-center management" and "Ethical Management" were extremely low, which indicated that theoretical structure of "mid-term evaluation " was not applicable to data analysis of "process assessment".2. The optimized quantitative indicators of process quality control evaluation for TCM clinical research consisted of 6 essential factors (research process, research records, authenticity and compliance, electronic data management, sub-center management) and 14 indicators (number of included cases/total number of cases, number of completed cases/total number of cases, unreasonable alternation of medical records, integrity of medical records, accuracy and integrity of laboratory physicochemical examinations, verification of the truth of subjects, signatures of informed consent, expulsion rate, consultation rate, electronic data submission, reporting the electronic data within the specified time window, consistency check between the electronic case report form (CRF) and key indicators in medical records, primary examination and secondary inspection).3. Results of the reliability analysis indicated:The combination reliabilities of "electronic data management" and "sub-center management" were up to 0.9, and the combination reliabilities of all latent variables were over 0.7, demonstrating that latent variables could be effectively measured by corresponding observation variable indicators. Most average variance extracted (AVE) values of latent variables were higher than 0.6, except for "medical records" (close to 0.5), which illustrated that majority variation of latent variables are from real variation, and the measurable variables could strongly explain the variation of latent variable. Aside form a coefficient of "authenticity" and "compliance" (close to 0.35), most Alpha coefficients of latent variables were much higher than 0.7, indicating favourable internal consistency.4. Results of the validity analysis showed:The value of X" was 776.2 (sample size is 475), degree of freedom was 72, within acceptable range. Goodness-of-fit index (GFI) was 0.832, normed fit index (NFI) was 0.729, comparative fit index (CFI) was 0.747, thees-three indicators were close to 0.9, indicating good model fitting effect.5. According to the scores calculated respectively on the subjective weighting and objective weighting, it was favorable to distinguish high-quality projects and poor-quality projects on the two extremities (the first quartile and fourth quartile). The coincidence rate was 80%. For those projects in the middle (the second quartile and third quartile), the total coincidence rate was 80%. Only the 3rd and 9th subject share the same ranking, others were different.Conclusion:1. Construction, adjustment and optimization of quality control evaluation indicators on the basis of quantitative indicators of mid-term evaluation, relied on existing research foundation and consideration of data characteristics during different stages, which was the development and enrichment of the preliminary work.2. The results of reliability and validity analysis were within acceptable range, which indicated close internal relationship between the index construction and process quality control evaluation, this set of indicators was an effective tool to measure the level of process quality. It was feasible to apply statistical methods which were usually used for educational evaluation and psychological evaluation to TCM clinical research evaluation.3. Ranking the projects on scores calculated by subjective weighting of expert opinions and objective weighting of mathematical characteristics (PLS), could better distinguish high-quality projects and poor ones (the first quartile and fourth quartile). But it was hard to differentiate those between them.4. As we ranked the projects on scores, the order differed greatly although the scores were similar. The function of process quality control evaluation for TCM clinical research was to differentiate the level of the process quality, not only to allow sorting, furthermore, the result calculated on subjective or objective showed consistent conclusion in distinguishing high-quality projects and poor-quality projects, so it was advised to evaluate the project by classification method in the future.5. The purpose of evaluation was not simply ranking or differentiating the quality, but to guide and encourage the projects to move towards the right direction and goal. Conducting evaluation with indicators, the criteria and the content had shown good guidance function, and led the projects approaching the target. Therefore, evaluation of process quality control aiming at key links in TCM clinical research, had made positive impact on the improvement of research quality.6. Several methods were adopted in process quality control evaluation for TCM clinical research, including expert experience judgment and mathematical statistics method. We analyzed the similarities and differences between different methods, making some helpful attempts in evaluation methodology. Meanwhile, determination of process quality control evaluation method corresponded to evaluation purposes, research stage and data characteristics, illustrating different research stage should adopt corresponding evaluation criterion and analytical approach, instead of mechanical copy, which was an important conclusion drawn by this article.7. The construction of process quality control evaluation index for TCM clinical research, could reflect the execution level of projects objectively and effectively, and realize quantitative evaluation of whole-process quality control standard of TCM clinical research ultimately, which provideed theoretical support and experience for similar research projects in quality control evaluation.