| Intelligent text grading is the task of judging whether the knowledge expression contained in the student answer text is consistent with the given reference answer text.The realistic intelligent grading faces the few-instance sampling constraint problem,i.e.,only a limited number of student answers are selected to be labeled by experts as the calibration samples.These samples are the basis for intelligent model inference.Specialty texts contain specialized concepts such as terms and entities,and the language and syntax used to describe specialized knowledge is complex.So,it’s difficult to understand and infer specialized knowledge under few-instance sampling constraint.Existing mainstream techniques based on pre-trained language models learn generic knowledge in massive corpora.Prompt learning further reduces the dependence of models on samples during supervised learning of downstream tasks,but cannot address the complex knowledge contained in specialty texts.Therefore,studying highquality sampling and intelligent low-resource specialty text grading has theoretical significance for specialized knowledge understanding and reasoning.This paper studies intelligent text grading based on the national qualification examination,which has practical value in maintaining the authority and fairness of national examinations.The main contributions of this paper are as follows.(1)For the few-instance sampling constraint problem,this paper proposes the fewinstance sampling method based on data feature distribution.For specialized knowledge,students’ cognitive differences lead to a variety of forms of answer texts,and complex language,syntax and specialized expressions lead to difficult answer texts.In this paper,the few-instance sampling method based on data feature distribution is used to conduct few-instance sampling by measuring the differences and difficulties of student answers.We measure the differences of student answers based on the implicit features of text.We first analyze the main features of the TF-IDF vector of the student answer using principal component analysis to obtain the implicit feature vector.Then,we calculate the local outlier factor of the student answer to measure the difference.We measure the difficulties of student answers by calculating the difference between the two maximum probabilities in the probability distributions,when grading the student answer using the pre-trained grading model.Sampling by fusing difference and difficulty measurement to ensure that the few-instance sampling set covers more diverse expressions of specialized knowledge and difficult samples.This is not guaranteed by traditional equal-probability random sampling method.(2)For low-resource specialty text grading,this paper proposes the specialty text grading method based on data augmentation.For low-resource sample set obtained from few-instance sampling,we propose the data augmentation method based on text pair construction.It combines correct or incorrect answer texts from different samples to construct more inferential samples for data augmentation.For the large number of unlabeled student answers,we propose the pseudo-sample sampling and self-training method based on multi-sample joint inference.For each unlabeled student answer,multiple correct answers are sampled from the low-resource sample set,and multiple inference is performed using the grading model to calculate the pseudo label and measure the confidence.Afterwards,high-confidence pseudo samples are sampled and added to the low-resource sample set to self-train the grading model.Repeat the above process for multiple rounds to get the final grading model.The data augmentation method based on text pair construction fully exploits the inference information contained in the low-resource sample set.The pseudo-sample sampling and self-training method based on multi-sample joint inference uses pseudo-samples to augment the low-resource sample set.Compared with the traditional self-training method which only infers once,our self-training method can alleviate the error accumulation problem in self-training.(3)This paper conducts experiments on a real national qualification examination dataset accumulated over many years.For the few-instance sampling problem,we verify that the fewinstance sampling method based on data feature distribution is better than the traditional equalprobability random sampling method.For the low-resource specialty text grading,we verify the effectiveness of the specialty text grading method based on data augmentation in the lowresource scenario.We verify that compared with the traditional self-training method,the selftraining method based on multi-sample joint inference can alleviate the error accumulation problem.Finally,we verify the knowledge transferability of the pre-trained grading model,the effectiveness of the data augmentation method and self-training method in the few shot grading experiment. |
