Research On The Construction Of Evaluation Index System Of Medical MOOC

Background and Purpose: Since 2012,many MOOC platforms represented by Coursera,ed X,and Udacity in the United States have developed rapidly,making MOOCs gradually popular as an educational form.The application of MOOCs in higher medical education is very extensive,and there are numerous studies on medical MOOCs,which are growing rapidly.Building a scientific evaluation system for medical MOOCs is of great significance for improving the quality of medical MOOCs,enriching medical course resources,and promoting the reform of medical education and teaching.Currently,there are a certain number of MOOC evaluation studies at home and abroad.Countries such as Spain and Germany already have relatively mature evaluation standards for MOOCs.However,foreign MOOC evaluation standards are more suitable for courses launched on mature profit-making MOOC platforms,and have not been refined according to different disciplinary characteristics and requirements.In 2002,the Education Information Technology Standardization Committee of the Ministry of Education formulated the "Network Course Evaluation Specification(CELTS-22)",but there is currently no unified evaluation standard for MOOCs.At present,research on the construction of MOOC evaluation index system in China is mainly based on different theories,documents or actual recording experience of MOOCs to determine the basic content of MOOC evaluation.The analytic hierarchy process(AHP)or fuzzy comprehensive evaluation method is applied to determine the weight of indicators.However,most studies have not considered different disciplinary characteristics and requirements,and there are no studies on the construction of a medical MOOC evaluation index system suitable for undergraduate and graduate medical students.Based on the characteristics of higher medical education in China,this study attempts to construct an evaluation index system suitable for medical MOOCs(theoretical courses)in our country,providing a reference for the evaluation of medical MOOCs.Main research content and steps: Based on Taylor’s evaluation theory,total quality management theory,and scientific educational management theory,this study constructed a medical MOOC evaluation theoretical model.On the basis of the literature research on MOOC evaluation at home and abroad and the construction of medical MOOCs in China,the first-level indicators of medical MOOC evaluation were selected through literature research under the guidance of the medical MOOC evaluation model.Key elements of medical MOOC evaluation were extracted through case studies,and the Delphi method was applied to construct the medical MOOC evaluation index system.(1)By applying the literature analysis method,this study systematically understood and mastered the MOOC evaluation standards and research status at home and abroad,analyzed the basic situation of medical MOOC construction in China,summarized the existing problems and future development trends at the current stage.Based on the research content,105 relevant literature were classified into four categories: MOOC teaching quality evaluation research,MOOC evaluation tool research,MOOC evaluation method research,and MOOC platform evaluation research.The characteristics and laws of MOOC evaluation were compared and analyzed.This study systematically researched 1208 medical MOOC literature in China,summarized the research hotspots of medical MOOCs at the current stage,the main types of medical MOOC construction,and future development trends.The medical MOOC evaluation model was constructed through theoretical research,and the dimensions of medical MOOC evaluation were determined.17 literature on the construction of MOOC evaluation index system were thoroughly studied,and based on the content of the index system,the primary indicators were combined and classified into five categories: teacher team,course content,teaching design,teaching effectiveness,and technical means.The first-level indicators for evaluating medical MOOCs were selected.(2)By applying the case study method,this study extracted the key elements of medical MOOC evaluation.Four high-quality medical MOOCs suitable for medical students’ course learning on four MOOC platforms,including Coursera,ed X,Xuetang X,and China University MOOC,were selected for case studies.The courses studied were Anatomy offered by the University of Michigan,"Human Anatomy: Musculoskeletal Cases" offered by Harvard University,Medical Parasitology offered by Tsinghua University,and Pathology offered by Huazhong University of Science and Technology.Their course introductions,learning methods,teacher teams,course content,evaluation methods,and interface operations were analyzed to extract the key elements of medical MOOC evaluation as a reference for the content of medical MOOC evaluation indicators.(3)By applying the expert consultation method,the initial construction of the medical MOOC evaluation index system was completed.Based on the results of literature and case studies,the content of medical MOOC evaluation indicators was initially defined.Six experts in medical education management and medical MOOC production were selected for on-site consultation.The initial construction of the medical MOOC evaluation index system was completed based on the modification opinions of the experts.(4)By using the Delphi method and analytic hierarchy process(AHP),the revision of the content of the medical MOOC evaluation index system and the determination of the weight of each indicator were completed.A consultation questionnaire was prepared based on the initial draft of the index system and was sent to 18 experts in medical education management for three rounds of consultation.The content of the medical MOOC evaluation indicators was further revised based on the questionnaire results,and the weight of each level of indicators was determined.Finally,the medical MOOC evaluation index system was successfully constructed.Research results: The medical MOOC evaluation index system was constructed,which includes five primary indicators: teacher team,course content,teaching design,teaching effectiveness,and technical means.It also includes 13 secondary indicators with their respective weights and 26 specific indicator points.The teacher team includes three secondary indicators: reasonableness,professionalism,and advancement,with six specific indicator points;course content includes three secondary indicators: scientificity,completeness,and ideological and political education,with six specific indicator points;teaching design includes three secondary indicators: teaching objectives,interactivity,and appropriateness,with six specific indicator points;teaching effectiveness includes two secondary indicators: learning behavior and learning outcomes,with four specific indicator points;technical means includes two secondary indicators: applicability and controllability,with four specific indicator points.The weight of the five primary indicators are 0.27,0.26,0.15,0.18,and 0.14 respectively.The authority coefficient of the experts is 0.794,indicating a high level of expert authority.The Cronbach’s alpha of the two questionnaires used for index content revision were 0.822 and 0.910,respectively,both greater than 0.7.The validity indices of the entries were 0.974 and 0.923,respectively,indicating that the questionnaire has a certain degree of reliability and validity.The Kendall’s coefficients of the two rounds of expert consultation were 0.236 and 0.303,with P<0.001,indicating a high degree of coordination among expert opinions.Innovations:Firstly,a medical MOOC evaluation theoretical model was constructed based on Taylor’s evaluation theory,total quality management theory and scientific educational management theory.This model provides guidance for the construction of medical MOOC evaluation index system.Secondly,the medical MOOC evaluation index system was constructed including five primary indicators,13 secondary indicators with their respective weights and 26 specific indicator points.The construction of the index system is scientific and reasonable,which further enriches the management tools of medical education.Overall,the innovation lies in the integration of various theories and the comprehensive consideration of multiple factors in the construction of the medical MOOC evaluation index system,which will provide valuable guidance for the development and improvement of medical MOOCs.