Study On The Index System Of Phacoemulsification Assessment In Guangdong Province

ObjectiveWe establish an index system of Phacoemulsification (PHACO) assessment of Guangdong Province, which as an example of the second grade of medical technology assessment on the base of studying the health policy and theory of health technology assessment. Not only can this index system provide reference to improve PHACO technical level and management level for hospitals, but also it can support for health decisions of health administration departments. Otherwise, it can be reference for establishment of other second or third medical technology for sake of its regular application and medical safety.Research content1 Establish PHACO assessment frameworkThrough literature review, we analyse the theoretical research and application of domestic and foreign medical technology assessment and study on relevant policy documents. PHACO assessment framework was built by considering of the present situation of medical technology assessment.2 Build an index system of PHACO assessment2.1 Evaluation primaries:Review papers about the research and study the policy named "The management of Clinical application of medical technology" announced by Ministry of Health and "PHACO technology management (Trial) of Guangdong Province" announced by former Guangdong Provincial Health Department. Then break it down into many items on the principle of scientificity, comprehensiveness, operability, independence and comparability. After the research team’s discussing and revising, a draft of the index system is established. The assessment system is divided into 5 first grade indexes,17 second indexes,47 third indexes and 38 fourth indexes. First grade indexes are technical implementation requirements, safety, effectiveness, economy and social adaptability. Qualified agencies, personnel requirements, equipment requirements, management systems are some of the indicators in the second grade.2.2 Build up the evaluation system. Firstly, do further revisions and additions based on three rounds of Delphi. Secondly, calculate the arithmetic mean, coefficient of variation and full marks ratio of each index, and set the sector for index screening. Then simulate the assessment and grade each hospital. The data collected is analyzed statistically and correlatively and index of little correlation with dimensions it belongs to are deleted. Finally, the index system is determined.2.3 Confirm weight of each dimensions:Expert evaluation method is adopted to determine the weight of first grade. While percentage weight method and product method are used to calculate the weight value of each index on the second grade.3 Reliability and validity of the index systemTrain seven staffs who were participate in the audit of the PHACO and randomly selecte 55 hospitals for PHACO assessment. Then we simulate assessment by using the index system of PHACO assessment. After that, we test the internal consistency reliability by Cronbacha. At last, we test the content validity through literature research and expert opinions and test construct validity of the indicator system using factor analysis.4 Empirical application of the index systemBy simulating assessment we obtain average scores of 55 hospitals, which as a criterion to judge their level and analyse the clinical application of PHACO of Guangdong Province. Then calculate the consistent rate and compare to Expert audit results.Methods1 Literature analysisFor one thing, we use CNKI database, Wanfang, VIP journal databases, PubMed, ScienceDirect, EBSCO databases and other Chinese and foreign websites to collect literatures. For another thing, we learn the status of domestic and foreign health technology assessment and review the status and problems of domestic medical technology. We have to be familiar with regulations related to medical technology accessment. Besides, we need to learn how to establish the index system of medical technology assessment and selecte assess items on this basis.2 Case study methodThe first phacoemulsification machine in the world is developed by Kelman successfully in 1966. So far, PHACO has become an advanced treatment of cataract all over the world.90% of cataract patients in developed countries use this treatment. PHACO’s safety for the treament, good efficacy and wide usage and so on are representation in the second class of medical technology.Therefore, this study intends to use PHACO as an example to build the index system.3 Dephi MethodDelphi is to collects experts’ suggestions by anonymous questionnaire. After 2 to 4 rounds of consultation, summary and modification, most experts’ opinion would converge, which is as basis for prediction or decision making. Delphi is anonymous, statistical and responsive. In this study, three rounds of Delphi are taken for further supplement and screening.4 Mathematical Statistics4.1 Full mark ratio is the ratio of the number of experts who give full mark to the number of experts who participate in this evaluation. The range of full mark ratio is 0 to 1. Larger the proportion of specialists giving full mark to the indicator, more important is the indicator. Observation arithmetic average reflects the value of data group with the same quality. Lager the arithmetic average, more experts believe that is the important indicator. Coefficient of variation is defined as standard deviation divided by arithmetic average. It ranges from 0 to 1. Smaller of the coefficient of variation, more consistent are experts’opiniones.In this study, we determine the critical value and screen indicators with a method of the united application of full mark ratio, arithmetic average and coefficient of variation. The first 15 percentile P15 out of all indicators is set as the critical value of full mark ratio. The first 15 percentile P15 out of all indicators is set as the critical value of arithmetic average. The 85 percentile P85 out of all indicators is set as the critical value of Coefficient of variation. Then, combine three critical value and remain indicators whose full score is bigger than the first P15 of it and the arithmetic mean who is bigger than the first P15 of it and the coefficient of variation who is smaller than the P85 of it.4.2 The correlation coefficient is used to describe the correlation degree and interrelational form between indicators.In general, the correlation coefficient between indicators and dimension it belongs to should be greater than it is with other dimensions. Therefore, we exclude rearward items in its dimension by calculating correlation coefficients between index and its dimension.4.3 Reliability means that if measurement issues can be measured by measurement tool stably. It refers to the consistency or stability of the measurement results. Internal consistency of index system is analysed by calculating Cronbach a. Validity means that if measurement tool can measure characteristics of things accurately. Construct validity of index system can be measured through factor analysis.Results1 After three rounds of Delphi consultation, an index system of PHACO assessment of 5 first grade indicators,11 second indicators and 20 third grade indicators is constructed. Technical implementation requirements, safety, economy, effectiveness and social adaptation in the last round of research are weighted with 0.305, 0.258, 0,215, 0.120 and 0.103.2 30 experts are involved in this survey. They work in Guangdong General Hospital, Sun Yat-Sen University, Soutem Medical University, Guangzhou Medical University and their affiliated hospitals. They research in ophthalmology,medical administration, statistics and ethics. All of them are undergraduate or have higher education and they are in secondary senior or higher positions. Reliability analysis shows that three roounds of positive coefficient are 80.00%, 90.00% and 92.59%.Experts’ authority coefficient is range from 0.710 to 0.921. Coefficient of concordance were 0.400,0.395 and 0.278, P <0.01. All of these show the high degree of experts’ reliabUity and desirable results of the consultation.3 In order to test the reliability and validity of the index system,we sent out 385 questionnaires and recover all of them. 380 of the questionnaires are valid and the effective rate is 98.7%. The standardized Cronbach a is 0.837. The index system can be considered has good content validity for many reasons. Firstly, the index system is established on the base of PHACO’s features and technical assess requirements. Secondly, Delphi method, the arithmetic average, coefficient of variation, full mark ratio and correlation coefficient are used for screening, improving and revising indexes. Thirdly, experts participate in this resrearch are in various fields, such as ophthalmology, medical administration, ethics and statistics. They can make the index system more comprehensive and describe characteristics of PHACO assessment accurately. Otherwise, construct validity of the test show KMO= 0.704 and Bartlett ball inspection result is P<0.01, which mean that data are suitable for factor analysis. There are eight principal components whose eigenvalues are greater than 1 and the cumulative contribution rate of 73.11% of the total variance. The research finds main problems of hospitals are low submitting rate of equipment qualification and mainly medical staff qualification. And some of medical staffs are not implementing the institution of checking, maintaining equipments and comply strictly with the registration system and three-level ward round system. Besides, some of the medical record quality is poor and so on. The corresponding rate between simulation results and expert assessment is 94.55%.Conclusions1 It is reasonable and practical to use Delphi method to construct the index system of PHACO assessment.2 Qualitative consultation and quantitative indicators screening methods improve the scientific of questionnaires.3 Reliability and validity of the index system and empirical application prove the feasibility of its practical application. Medical staff should implement the institution of checking, maintaining equipments and comply strictly with the registration system and three-level ward round system. They should pay attention to preoperative examination, preparation,operation records and improve the quality of medical records.