Research On Construction And Application Of Multimodal Curriculum Knowledge Graph

With the rapid development of artificial intelligence and people's emphasis on education,more and more attention has been paid to innovative teaching and learning methods.In the university classrooms,teachers' teaching methods are no longer traditional blackboard writing,but have been transformed into multimedia teaching including text,courseware,video,speech and other forms.However,the general framework of curriculum knowledge is mainly presented in the form of catalog chapters in the textbook,but the small knowledge points contained in the chapters and the relationship between the knowledge points are not clear at a glance,which is not conducive to learn for student.Therefore,how to effectively and automatically integrate various forms of curriculum resources and better organize and present knowledge have become important research topics.Facing the above problems,the knowledge graph provides an effective solution.This article takes a university computer major data structure course as an example to construct a multimodal curriculum knowledge graph.On the one hand,it integrates data from different sources and different structures,extracts the entities contained in the data and the semantic relationship between entities,and provides a knowledge management model.On the other hand,the knowledge points are displayed in the form of an intuitive and clear knowledge graph,which is beneficial to the user's learning of the knowledge points.The multimodal curriculum knowledge graph constructed in this paper makes the text and speech modal curriculum resources complement each other,realizes multimodal resource integration and the retrieval of the lower-level knowledge point learning path of the course,helping users to learn efficiently.The main contents and contributions of this article are shown as follows:1.This paper defines the data model of the multimodal curriculum knowledge graph,that is the definition of entity types and relationships between entities.Knowledge point entities are defined as three types of structure,algorithm,and related terms,speech is defined as entity type,and six semantic relationships are defined between entities.2.This paper proposes a named entity recognition model that combines bidirectional long short-term memory networks and convolutional neural networks.It combines local and global information,uses a conditional random field model to label feature sequences,and combines a domain dictionary.It makes the model accuracy rate and recall rate have been improved.The comparative experiment proves that the model has a better named entity recognition effect.3.In the work of relation classification,this paper combines lexical and syntactic feature vectors as the input of the classifier model,and uses bidirectional long short-term memory networks based on attention mechanism to process the relation classification task.Compared with other models,it proves that the model performs best,and a visual analysis of the attention mechanism to improve the effect of the model is performed.4.This paper proposes a multimodal entity linking method,which mainly solves the multimodal problem in knowledge graph.Firstly,the speech of the teacher's lecture is recognized as text by speech recognition technology.Secondly,the matching link between the speech and the knowledge point entity is realized by the text matching algorithm,and the relationship between the two is defined as the association to complete the multimodal entity linking work.The experimental results show that the named entity recognition algorithm and relation classification algorithm in this paper have better results,which can make the constructed knowledge graph have higher quality.This article finally builds a complete multimodal curriculum knowledge graph based on the Neo4 j graph database to store knowledge.In order to explore the multimodal curriculum knowledge graph constructed in this paper,it has a good application value.This paper builds a multimodal retrieval platform for curriculum based on knowledge graph,provides a multimodal search case based on knowledge points,and guides students to learn through the relationship between knowledge.At the same time,it can provide theteacher to explain the speech of specific knowledge points in the classroom,mobilize students' multiple senses,and improve the learning effect of students.