The Research And Application Of The Quality Evaluation Of Large-scale Examination Based On Deep Learning Technology

Large-scale examinations usually refer to examinations that are organized and managed by professional examination institutions within a country or a larger economy,with the main purpose of selection,evaluation,and monitoring.In my country,large-scale examinations mainly refer to national examinations with important influence such as college entrance examinations and postgraduate examinations;internationally,large-scale examinations include some well-known examinations and evaluation projects such as SAT and PISA.This article focuses on the large-scale examinations in our country,especially the college entrance examination.There is a fundamental difference between large-scale examinations and general examinations.Due to the small test takers of general examinations,its influence is small,and there are relatively few restrictions from various aspects.Large-scale examinations have higher social attention,higher quality requirements,and greater social impact.Especially,the large-scale examinations have formed a set of relatively fixed models to ensure the quality of test questions.These traditional models have played an important role for a long time.However,the larger the scale of the test,the more difficult it is to change.Now the traditional models are undergoing a severe impact,and only changes can be made to cope with various challenges.How to solve such an urgent form and complicated dilemma has become an important and urgent task.Deep learning technology provides important support for changing and solving this dilemma.How to achieve the deep integration of deep learning technology and large-scale examination has become an important topic for improving the quality eval-uation method of large-scale examination questions,which has great theoretical and practical significance.However,the current application of deep learning technology in examinations has just started and is in the exploratory stage.Its role in some specific examination tasks has not been fully utilized.The empowerment of examinations still needs to be systematically sorted out and explored in depth.In this context,this pa-per proposes several methods based on deep neural networks to solve problems such as difficulty prediction and similarity judgment in large-scale examinations,and is com-mitted to putting them into practice.The work and contributions of this article can be summarized as follows:First,based on the deep representation framework,the difficulty of the test ques-tions is automatically estimated.The difficulty prediction of test questions is one of the core problems in large-scale examinations.Neither difficult nor easy can accurately measure student abilities.There are two traditional ways to predict the test question difficulty:artificially estimated by expert experience and pre-test.Pre-test is that the test questions are opened to some students to calculate the difficulty of the test ques-tions by analyzing the answer data from the students.The difficulty of test questions estimated by experts is often highly subjective and unstable;pre-test can easily lead to test questions leaking.In particular,the two methods are not related with each other,and the data of the test questions(content and answers)are not unified.This research is based on the deep representation framework,deeply mines the test question text data combined with a large number of student response data,anlysises the relationship be-tween data and quality,so as to realize the automatic prediction of the question quality.Specifically,after word segmentation,each question is expressed as a vector set of dis-tributed representation,and then CNN and RNN models are introduced to obtain the local information and sequence information of the test questions.In the loss function,the sample dependence of the student's response data is eliminated through the pairwise form,and the model training is carried out to obtain the relationship between the test question text and the test quality parameters.Finally,through the task of predicting the difficulty of the test questions,the correctness and effectiveness of the method are verified.Secondly,this dissertation combines the attention mechanism to mine the relevant information between the different structures of the test questions,and deeply under-stands the semantics of the test questions.The previous work mainly studied the single-form test questions,which are mainly mathematics questions.However,some reading questions are complicated.In addition to considering the test questions themselves,the difficulty of the test questions also needs to be studied in combination with the mate-rials on which the test questions are based,the options of the test questions,and other information.In order to achieve the quality evaluation of such test questions,innova-tive research methods are needed.On the basis of the previous work,this dissertation introduces attention mechanism to search for deep-semantic information that affects the quality of the test questions,and discover important parts of the test question materials related to the question,so as to improve the accuracy and stability of the feature min-ing.Specifically,the model structure consists of four parts,the input layer,the two-way GRU layer,the attention layer and the prediction layer.Among them,the two-way GRU layer can not only learn the remote dependencies of the entire input sequence,but also learn context information from the forward and backward at the same time;the attention layer uses Attention mechanism,which can extract words in documents or options that are more relevant to the test questions.As the main information for the estimation of the difficulty of specific test questions,it helps to visualize the model and improve its interpretab ility.Thirdly,this dissertation combines the knowledge structure to explore the deep re-lationship between the test questions.The study of correlation between test questions is another core issue in the large-scale examinations.In the past,the relevance of test questions relied heavily on expert prediction.This method was highly subjective,es-pecially when faced with large-scale test tasks,the efficiency was not very high.Some researches on the relevance of test questions based on shallow semantic information have not achieved the expected results.This research is based on the previously de-signed representation framework,combined with the knowledge structure of the test questions,and then merged with the representation of the test content,so as to achieve a more comprehensive test question representation to judge the deep semantic relationship between different test questions.Concretely,we take advantages of ContentRepresen-tation Layer(CRL)and Structure Fusion Layer(SFL)to respectively handle the exercise content and the knowledge concepts with the knowledge structure.In CRL,we focus on learning a unified semantic representation of the exercise content including texts and images.We first use the embedding method and a Convolution Neural Network(CNN)to represent texts and images.Then,an Attention-based Long Short-TermMemory net-work(ALSTM)is utilized to model the inner asso-ciation of text-image.In SFL,we put stress on characterizingthe knowledge structure.Specifically,we first model the inner association of content-knowledge by using an attention-based block,namely Content Knowledge Attention(CKA)block.After that,a Tree Convolutional Network(TCN)is designed to model the association among knowledge concepts in a bottom-top man-ner on the tree-like knowledge structure.By exploiting the knowledge structure,SFL can not only retrieves the structure-aware semantic representation but also provides an interpretable view to investigate the similarity of exercises.Finally,we adopt a Simi-larity Score Layer(SSL)and conduct a pairwise training strategy for returning similar exercises.Extensive experiments on real-world data demonstrate the effectiveness and interpretability of KnowNet.Finally,the practice and application of research makes remarkable achievements in large-scale examinations.The research had been deeply and concretely integrated in the system.The model parameters obtained through the difficulty estimation algorithm training are integrated,and the estimated difficulty corresponding to each test question provides an important data reference for experts.The automatic labeling of test item attributes has also been integrated into specific work.The result of similar question judgment can provide feedback on the same or similar test questions in the system for any new question,and provide reference for experts.All these research results have been integrated in large-scale question banking system,which have played an active role in the examination industry,and have greatly affected the digitalization process of large-scale examinations.Based on the results of this research,a number of system-atic projects related to large-scale examinations have been rapidly developed,such as standardized management of test resource resources,test semantic query and retrieval,test crawlers and test materials collection,etc.The results of this research explored the practice path of the integration of large-scale examinations and deep learning,and provided solutions,which have been highly valued by management departments and widely recognized by the society,and play a important role in serving and ensuring the safety and stability of national examinations.