Research On Multimodal Knowledge Tracing Methods Based On Heterogeneous Graphs

The adaptive education system pursues the goal of customizing personal learning proposals and providing targeted teaching resources in the online teaching environment according to the historical learning conditions of students,which is a hot topic in today’s education technology.Knowledge-tracing technology is the core and key to building an adaptive education system and is often applied to grasp students’ knowledge-acquiring status and predict their future performance.Existing studies have gained fruitful results for knowledge-tracing models but still have faced some limitations.(1)In the existing studies,the student-question topological structure and the influence of students’ learning ability are rarely discussed.In addition,there is a lack of research on the importance of the problem.(2)The existing models of questions and skills are built solely based on structural information,so it is impossible to capture the interdependencies of questions and skills using multimodal information about them.The existing models quantify the student’s memory levels only from the time dimension and ignore the effect of different modalities on the memory level.(3)The existing models can only quantify students’ overall knowledge status to predict whether students can give the correct answers in the next step but cannot recommend courses according to the knowledge mastery level of students.To address the above-mentioned three limitations of knowledge tracing models,the following research work was done.First of all,a SPKT(Student-Question Knowledge Tracing)was built based on a heterogeneous graph to address the Limitation(1).This research embedded student nodes according to students’ historical answer records,calculated students’ learning abilities and took the calculated learning abilities as the attributes of student nodes.Meanwhile,this thesis embedded question nodes according to the ability-question mapping,calculated the question importance by using the Page Rank algorithm,and took the calculated question importance as the attributes of question nodes After that,the graph attention network incorporating a multi-headed attention mechanism was used to deliver and fuse information on the student-question relationship graph,and the edge classification was taken as a training task to predict the future questionanswering performance of students.Compared with the existing optimal model,this model was improved by about 1.0% in AUC and about 2.9% in ACC.Furthermore,a multimodal knowledge-tracing model incorporating the forgetting mechanism was established to cope with the Limitation(2).This research trained the model by matching question and skills nodes with the graph to optimize unimodal embedding.The similarity between nodes after multimodal fusion was calculated to get the question-ability relationship weight,and the question-ability mapping and the question importance were calculated to generate the question nodes.Moreover,this thesis constructed and used a long and short-term memory network to analyze the student knowledge status with forgetting factors.The student knowledge status was incorporated into the student answer records to generate student nodes.The strength of the student-question relationship was calculated based on the number of students’ answers and the effective memory rate of different modalities,and the graph attention network was used to transfer information and predict students’ answers to different questions.Four classification tasks(AUC,ACC,Recall,and F1)of two datasets were selected to measure and verify the effectiveness of the model built in this thesis.Compared with the comparison model,this model was improved by 1.91%～6.22% in AUC,0.13%～14.33% in ACC,0.58%～8.53% in Recall,and 0.08%～14.31% in F1.Lastly,a knowledge-recommending system incorporating the knowledge tracing model built in this thesis was established to deal with the Limitation(3).The knowledge-tracing model can help teacher to determine the student knowledge mastery based on the student learning and answer records.This system can also push exercises and courses to students.Besides,this model can collect student history records to provide data support for subsequent research.