An Interpretable Method Of Essay Grading Based On Expert Knowledge

Writing ability is an important index to evaluate student language level in Chinese education.Automated essay scoring can not only provide technical support for the objectivity and impartiality of manual evaluation in the field of examination and evaluation,but also help to improve personal essay ability,which has important theoretical value and application value.Automated essay scoring still faces many challenges.First,the consistency of expert scoring indicates that there are rules to find in the essay-scoring task,but there is still a lack of quantifiable scoring rules.Secondly,as a symbolic system,language is more flexible,the processing of natural language is more difficult than that of structured data,and the process is less interpretable.Finally,considering the problem of corpus labeling,most of the existing essay grading work focuses on giving an overall score to the essay,and there is a lack of fine-grained labels for essays.Vocabulary is a basic component of essay and an important scoring basis.It is very necessary to quantify the essay level from the perspective of vocabulary and integrate it into the modelAiming at the above problems,this paper started from the aspect of the essay vocabulary usage,and proposed an interpretable essay grading method based on expert knowledge.The research content of this paper includes the following three aspects(1)We proposed a quantifiable evaluation framework of Chinese essay vocabulary based on the expert review criterion,and marked the vocabulary level of corpus composition.In order to improve the interpretability of the model from the perspective of vocabulary level,we analyzed the composition corpus of primary education,primary school and middle school,studied the scoring standard of college entrance examination and several vocabulary banks of grades,and formed a hierarchical vocabulary list based on expert knowledge and statistical analysis.Based on the hierarchical vocabulary,combined with the analysis and statistical results of the corpus,we put forward the evaluation rule of computable essay vocabulary level,and marked the vocabulary level of cach sentence in the corpus with the evaluation rule.(2)We studied interpretable essay grading methods based on deep neural networks.In order to explore the implicit logical and semantic relations between the words and sentences,we modeled the essay,and used a Bi-LSTM network based on the word embedding to generate the essay representation vector and give the composition rating.We introduced the attention mechanism into the lexical and sentence levels of the model,extracted the high score sentences in the essay,and proposed a novel interpretable method to explain the model from the specific aspect of the essay vocabulary level.In order to prevent from using the model scoring rules to obtain high scores,we analyzed the robustness of the model,and added the syntactic detection to the model.(3)We verified the performance of the model on the real corpus.Aiming at the essay corpus of primary and middle schools,we verified the performance of the model on the essay-grading task.Compared with state of the art methods,our model is more consistent with the results of manual review and has higher interpretability.For a specific exam,we adapt our model to the adult college entrance examination in a province.Experiment shows that our model can also give accurate scores to the essay in a specific exam.