| Objective(1)To know about the defects of test paper making by analyzing the test paper and items , and to seek the methods to improve the quality of test paper, so that made the test papers preparing from empirically to theoretically ; (2)To find the problems and weakness of teaching , so that provide a basis for education reform in future ; (3) To make more and more people learn, understand and apply education measurement; (4) To build and update the item bank and give the guidance to develop the test paper analyzing system and accelerate the pace to set up test center.MethodsThe literal research for papers and manuscripts published domestically and abroad about education measurement is the theoretical guidance and academic background for the research.The resource of item and the methods of paper making are available by calling and interviewing.Calculate the reliability, validity, difficulty and discrimination coefficient by Excel. The valid data are input electronically and the database in constructed using SAS9.1.3. The software package SAS9.1.3 is utilized to implement the data analysis when double check is made for entry.ResultsThe quality of the test papers which conclude the courses of Pathology, pharmacology, obstetrics and gynecology, internal medicine for 5-year clinical students of Grade 2003-2005 in Nanjing Medical University were analyzed by applying education measurement. The statistics of the examination papers had shown the following result.(1) The scores of all the papers showed negatively skewed distribution. the skewness of 41.67% test papers were above "-1" while only 15.38% were within "0.5" .In general ,the trend of skewness had become small ,except Pathology.(2) Cronbach coefficient alpha of all papers had reached 0.6,and 91.7% percent were above 0.7 except obstetrics and gynecology which is 0.68,which was in an acceptable range .The reliability of internal medicine was higher than the other three curriculums ,Cronbach coefficient alpha of which were all above 0.8 in the three years. However,only 41.7% of the papers reached 0.8,and not any paper was 0.9 or above.(3) All the content of the chapters had been included in the test paper,but the distributions of which were not consistent with the teaching programs.(4)Both objective and subjective questions and issues had been used to measure,but the proportion were different. From Grade 2003 to 2005,the ratio of Pathology were 60:40, 80:20 and 60:40 respectively;Pharmacology were 65:35, 65:35, 70:30 respectively;both gynecology and obstetrics and internal medicine were same ,the ratio of which were 30:70.(5) All the cognitive domain concluding "recalls ,interpretation and problem solving" had been measured in the papers,but the proportion were not agree with the principle of paper making. From Grade 2003 to 2005,the proportion of pathology were 74:9:17, 82:7:11 and 82:7:11 respectively;pharmacology were 85:13:2, 89:7:4, 89:6:5 respectively;gynecology and obstetrics were 39:25:41, 37:30:38, 40:32:33 respectively; internal medicine were 50:26:29, 52:32:21, 64:19:22.(6) The different demand of teaching program including master, familiar with and knowing about had been tested in all papers,but the proportion varied. From Grade 2003 to 2005,the proportion of pathology were 75:22:3, 84:15:1 and 84:12:4 respectively;pharmacology were 81:18:1, 72:22:5, 91:7:2 respectively;gynecology and obstetrics were 71:26:8, 68:19:18, 59:27:19 respectively; internal medicine were 70:26:9, 80:17:8, 74:14:17.(7) The difficulty coefficient of the four courses were high, which showed the test paper were easy,although it had added difficulties year by year. 41.67% of the papers,the difficulty coefficient was higher than 0.8,and Grade 2003 and 2004 of obstetrics and gynecology were 0.88 and 0.87 respectively. There was significant difference in difficulty in obstetrics and gynecology tests versus the other three curriculums,and internal medicine tests versus pathology and pharmacology ,which demonstrated the difficulty of obstetrics and gynecology tests were lower than pathology and pharmacology,which were also lower than internal medicine tests.(8) The discrimination coefficient of four courses are not high, and 75% of the papers are between 0.20-0.30,even 16.67% of the papers were less than 0.20.The increasing tendency of discrimination year by year had been revealed from the change of Grade 2003 to2005 of all the curriculum examinations. obstetrics and gynecology tests had lower discrimination than the three other curriculums in the distribution of item quality,while in the distribution of scores ,the four courses Paper presented that pharmacology was better than pathology, pathology was better than internal medicine, and internal medicine was better than obstetrics and gynecology .ConclusionAccording to the principle of education measurement, the scores of the papers which of the pathology, pharmacology, obstetrics and gynecology, internal medicine for 5-year medical students of grade 2003-2005 in Nanjing Medical University, demonstrated that reliability was high , difficulty was low, discrimination was not high,the validity of measurement needed to be improved. The difficulty and discrimination coefficient of the four courses which are the core curriculums of clinical medicine should be adjusted to the same level, although all the test paper have increased the difficulty and discrimination coefficient year by year. All the content of the teaching material had been included in the test paper,but the proportion was not coincided with the teaching program both in items and scores .Both the subjective questions (noun interpretation, short answer ,case analysis questions .et) and objective questions (multiple choice,et ),all the cognitive domain concluding memory, interpretation and problem solving,all the demand of teaching program including master, familiar with and knowing about were measured in the test,but the proportion was not based on the principles of test paper making, especially the pathology and pharmacology which standed for preclinical courses. In conclusion, there were some flaws in test papers , so we should increase the quality of test paper making because the method of measuring the effectiveness of student learning and teaching in this college was not scientific enough. The quantitative analyses and scientific evaluation by applying the method of education measurement can improve the quality of examination items and the validity of examination papers, which can provide a strong evidence for education reform. Meanwhile ,the scientific and rational evaluation of the effect of teaching can be available through the item bank building by the applying the method of education measurement . |
PDF Full Text Request