Study On The Use Of Computer-based Testing In General Practitioner Training

PrefaceIn medical education field, the great essencial differences exists between clinicalpractical competence and simple theoretical test in test contents, forms, andmanagement. However, traditional clinical practical test has many limits,including the huge work amount, the difficulty in quantifying the score standardand the unequal question difficulty. And it is extremely unsuitable for the largenumbers of and standard practical skills test of general practitioners (GP). GPtraining in our country started relatively late and the test system for GP is farfrom perfect. Exploration of objective and effective method for GP assessmentbecomes an important issue.Computer-based Testing (CBT) is a new test type and it includes method ofComputer-based Case Simulations(CCS), which presents an interactive,dynamic simulation of a patient-care situation and examinees can take actionsaccording to the changes of the simulated patients. CBT has been developed and used for decades aboard. In the United States, it has been widely applying intoMedical Licensing Examination, Family Medicine Post-graduated Education andTraining Examination, and so on. In our country, the National MedicalExamination Center and a few universities including China Medical Universityand Zhejiang University also have began to use CCS to evaluate clinical abilityon trial.In order to found an objective and effective method to evaluate the practicalskills of GP, a study focued on the research and development of CCS for GP testsystem and its efficiency to evaluate the GP' practical skills appropriately.Subjects and MethodsCCS for GP test system is coherent and dynamic. It imitates the work scene ofGP, and goes around the case and test targets. The simulation case constituted aseries of questions, dividing into modified essay questions and patientmanagement problems. Case design targets at common health problems incommunity. 134 in-service training GP, 32 resident training GPs and 8 GPs wereall tested by CCS. Reliability, validity, difficulty, discrimination andconfidentiality of CCS were all analyzed. Comparison had been done betweenmultiple SPs (Standardized Patient) station test and CCS test among 18 residenttraining GPs. Questionnaire survey was conducted among students, experts andGPs. The content involved case design structure, test form, case sampling andknowledge coverage. All the statistical analysis was carried out in SPSS 13.0 forwindows software. ResultsThe mean score and its standard deviation ((?)±s) in-service training GP group,resident training GPs group and common GPs group were 59.3±7.66,68.07±5.07 and 68.1±4.36, respectively. The alternate form reliability of CCSwas 0.723 measured with r_u (P＜0.01) and test-retest reliability method was 0.665measured with r (P=0.001). Questions whose discrimination ranged from 0 to0.19 accounted for 9.5ï¼….Tested same in-service training GP group by CCS with same examinationquestions in one year interval, Score means was compared by paired-samples Ttest. The Contents validity indicated results showed that score means betweentwo groups had statistical difference (t=-5.34, P_0.05=0.001).One-way ANOVA LSD method was used to compare the score means amongthree different in-service training GP groups, The results showed that scoremeans had no statistical difference (P=0.358～P=0.911).One-way ANOVA LSD method was used to compare the score means amongdifferent groups, of which the difference reached statistical significant level(P＜0.001) between in-service training GP group and resident training GP group,in-service training GP group and common GP group, but not between residenttraining GP group and common GP group (P=0.976).Score means of SPs station and CCS was compared by paired-samples T test.The results showed that score means between SP total clinical skill test andCCS had no statistical difference (t=0.38, P=0.709). Correlation coefficient of SPtotal test means, SP clinical skill test means of general medicine and CCS testmeans were between 0.475 and 0.586 (P=0.014～P=0.047).Taking SP station test results as independent factor and CCS test results asdependent factor, two models based on multiple linear regression were fittedsuccessfully. One was the SP clinical skill test and CCS (F=7.622, P=0.014). The other was the SP clinical skill test mean score, SP general subjects test and CCS(F=6.440, P=-0.010).Principal Components Analysis method was used to SP total test score, SPclinical skill test score, score of general medicine, taking saving variables asindependent factor and CCS test results as dependent factor, the model basedon multiple linear regression were fitted. (?): 24.6+0.361x1+0.137X₂+0.297X₃eliminate one abnormity student test score, as same analysis method, the modelbased on multiple linear regression were fitted. (?): 14.01+0.4092X₁+0.371X₂+0.3389X₃Proportions of students, experts and GPs considered the case structurereasonable were 85.6ï¼…, 66.7ï¼…and 75.0ï¼…, respectively, considered the testform appropriate were 83.8ï¼…, 100ï¼…and 87.5ï¼…, respectively, considered thecase sampling reasonable were 84.6ï¼…, 83.3ï¼…and 75.0ï¼…, respectively, andconsidered the knowledge coverage suitable were 85.6ï¼…, 83.3ï¼…and 62.5ï¼…,respectively.ConclusionsPractical application demonstrates that CCS test system has reliable function,and favorable confidentiality, reliability, validity and discrimination. Based onthe comparison results with SPs station test, we conclude that CCS test for GPexamination is effective and feasible, which can be used as complementarymethod for GP clinical skill test. Further design with standardization andobjectivity will be contribute to improve the precision and efficacy of the testsystem. The combination of SP test with CCS test into GP assessment will be more complete and objectivity.InnovationsOur paper reported the application of CCS into GP assessment and comparewith SPs examination. Analogous paper was not seen before this report in ourcountry. Our study makes main concept, basic principle, basic method ofgeneral medicine and community common health problems integrateorganically, and use computer to simulate medical treatment scene incommunity. In CCS system, GP test adopts the same standard and achievequantification and objectivity for score with computer technology.Multi-objective test guarantees that test result can comprehensively embodyexaminees skills in dealing with community health problems. Due to security,validity, saved manpower and material, CCS system also brings in niceeconomic and social benefit.