Design And Implementation Of Intelligent Teaching Assistant Platform Based On Multi-Modal Knowledge Graph

Knowledge Graph,a key technology to express the relationship between knowledge structures,has been widely used in various fields since it was proposed.There are complex structural relationships between multi-source and heterogeneous teaching data in education and teaching.It is of great research value and practical significance to construct the intelligent application of a multi-modal knowledge graph in education based on a knowledge graph and combined with various artificial intelligence processing technologies.The intelligent teaching assistant platform based on a multi-modal knowledge graph(hereinafter referred to as multi-modal teaching assistant platform)is a teaching assistant system that integrates multi-source video,voice,and text around multi-modal knowledge graph technology.Implementing the system completes the real-time supervision of the classroom learning situation through YOLOv3-tiny and builds a fine-grained knowledge point graph using Paddle OCR and Deep KE.By constructing a logarithmic probability regression model,the teaching video is automatically segmented,and the linkage and mapping of multi-modal data,such as fine-grained knowledge points,video teaching,classroom learning,etc.in the atlas are realized to complete comprehensive multi-modal applications such as "intelligent learning situation analysis","multi-dimensional quantitative assessment","knowledge atlas thermal map","personalized question bank generation" and "learning situation deterioration delimitation and positioning".Multi-mode teaching assistant platform is not a single mode of reasoning and application but a multi-model,multi-mode coordination work involving computing power,real-time reasoning,task scheduling,and many other issues.This paper introduces the construction of knowledge graph,multimodal teaching data processing,and intelligent teaching application construction and deployment.The main contents of the design and implementation of the multimodal teaching assistant platform are as follows:(1)Construction of course knowledge graph: based on the Deep K framework,complete the entity extraction of conceptual knowledge data to generate link anchors for multi-modal data.Label entity relationships according to the entity-relationship text data set.Load the entity relationship set into Deep KE training to generate the entity relationship extraction model.Analyze the result data of Named Entity Recognition(NER)and Relation Extraction(RE)in the test set to study the practical application effect of knowledge concept entities as graph nodes.(2)Multimodal processing and linking: In classroom teaching,there are two video lines,the video line for teachers to teach and the video line for students to listen to.Three modes are involved in the two video lines: multi-source video,text,and audio.The three modes are all around a fine-grained entity anchored within a certain time window,but the content expressed is different.The student video mode contains students’ learning action status data in class,and the teacher audio and video and multi-source text mode contain the in-depth description of an anchor.Then based on the constructed curriculum knowledge graph,the above multi-source heterogeneous information needs to be processed separately and then accurately linked and mounted to the corresponding knowledge anchor.(3)Design and implementation of multimodal teaching assistant platform: based on the anchor of the multi-modal knowledge graph,multimodal information is integrated to build an intelligent teaching assistant platform based on the multi-modal knowledge graph.It includes the design and implementation of multiple functions,such as the delimitation and positioning of the deterioration of the academic situation,the analysis of the thermal diagram of the academic situation,and the multi-dimensional quantitative assessment and evaluation.The construction of multi-modal teaching auxiliary platform is conducive to students’ sorting out the relationship of knowledge structure,simplifying the data processing process,helping teachers to monitor the dynamic situation data,quantitatively analyze teaching results,display visual learning data and develop personalized learning programs,solving the problems of data organization,quantification and display in the process of "teaching" and "learning",and providing specific application ideas and solutions for the implementation of smart education.