Statistical Methods And Application Of The Detection Of Answer Copying

In order to compare the power of answer copying detection of the answer copying statistics such as series of k-index, series of s-index, g2-index, false same rate, series of kappa statistics, and based on the shifted binomial statistical test indicatior, the comparison experiments with the real data of medical licensing examination in china were carried out under various copying condition, and the hypothesis of the experiments were the copiers and sources under suspicion were determined. Then the reason why the power of answer copying detection of kappa statistics was worse was investigated, and the improvement of kappa statistics was tried out. Simultaneously, considering the main style of cheating in china was inter-regional and high-tech group cheating with the copiers and sources under surpicion unknown, the comparesion experiments under the charateristics were also carried out. Finally, the "good" answer copying statistics with high power of answer copying detection were picked out and comprehensive applied on inter-regional and high-tech group cheating. The details were as follows:(1) The comparison experiments of the answer copying statistics based on the simulated data and the real data of medical licensing examination in china were carried out. The results indicated the following:among sample size, test length, source’s ability, and percentage of items copied, percentage of items copied was the most influential factor of their power of answer copying cetection, test length and the source’s ability were the second ones, and the sample size was the least one. Under all experiment conditions, the empirical type Ⅰ error rate of g2-index, false same rate, kappa statistics, and based on the shifted binomial statistical test indicatior were all above the nominal a level, and the detection rate of k*’-index was low. K1-index, k2-index and series of s-index were the best answer copying statistics when the copiers and sources under suspicion were known.(2) The reason why the power of answer copying detection of kappa statistics was worse was investigated, and the improvement of kappa statistics was tried out. The results indicated the following:obtaining the critical values of answer copying detection based on empirical distribution of the responses of all examinees and changing four-rank ordinal scale into ten-rank all could yield low empirical type I error rates. Kappa1statistics which only improved according to the first way and kappa2statistics which improved according to the two ways all could hold the empirical type I error rates below the nominal a level.(3) Considering the main style of cheating in china was inter-regional and high-tech group cheating, the comparesion experiment under the copiers and sources under surpicion unknown was also carried out. Of course, all of the answer copying statistics was taked requisite adjustment. The results indicated the following: comparing with the copiers and sources under suspicion known, the empirical type I error rates of all answer copying statistics were sharp increased; the comparative advantages of based on the shifted binomial statistical test indicatior were the best detection rates and convenient calculation; k*’-index and kappa2statistics were the best answer copying statistics. The nominal a level of k*-index and kappa2statistics should be respectively set to0.00003and0.001, and the detection rates of them were all above0.9when the sample size were200, and the percentage of items copied were above30%.(4) Taking medical licensing examination for example, a new system to detect answer copying was presented. The system combines two statistical approaches in successive stages. Based on the shifted binomial statistical test indicatior was used to compare all examinees in the first stage, and the nominal a level should be set to0.00003. K*’-index and kappa2statistics were used to compare the suspicious examinees picked out in the second stage. The parameter b of k*’-index should be set to0.4, and the critical value of k*’-index and kappa2statistics should respectively be set to0.00004and0.6375. Percentage of items copied was the most influential factor of the system’s power of answer copying cetection, and if it was above30%, most of the copiers could be accurately detected.