Cross-media Education Search Based On Sematic Fusion And Precise Research

In the background of the stupendous development of the Internet and artificial intelligence,big data has an impressive growth in people’s daily life.Meanwhile,the educational big data also has a place among them,playing a vital role in guiding students,teachers and colleges to search relevant information.With the explosive of big data different modalities of educational big data information have begun to emerge,more and more users put forward higher requirements to single-modal retrieval.To sum a word,for the sake of making full use of educational data of different modalities,it is indispensable to adopt emerging intelligent methods,such as deep contrastive learning and generative confrontational learning,to promote the cross-modal retrieval performance of educational big data and the accuracy of searching results.Staring from users’needs,the intelligence and efficiency of the cross-modal search engine for educational big data can be improved,resulting to more precise retrieval results.Therefore,in-depth research on cross-modal semantic fusion and precise search of educational big data is of great significance to the retrieval and application of educational resources.The work completed in this thesis mainly includes the following four aspects:(1)In terms of cross-modal semantic fusion of educational big data,a multi-modal resource acquisition method for educational big data is proposed in view of the existing online educational resources that have significant problems such as large noise and content redundancy.A crossmodal feature fusion model based on fine-grained semantic reasoning if proposed for the sake of the ignorance of fine-grained sematic information in the educational data field.When collecting cross-modal educational big data information,a thesaurus of educational big data is constructed,the preliminary information collection is completed through web crawler technology,and the basic resource library for the entire model training is obtained through data cleaning and classification.In the cross-modal feature fusion method based on fine-grained semantic reasoning,the cross-modality of educational big data is realized on the basis of the attention mechanism by learning fine-grained semantic information of image and text modalities in educational big data Semantic Fusion.Experimental results show that the proposed model has significant advantages in cross-modal search performance on educational big data.(2)In terms of cross-modal hashing learning of educational big data,a cross-modal hashing learning model based on multiple contrast and two-path antagonism is proposed to solve the problem that existing unsupervised hashing methods lack of fine-grained feature semantic association learning.On the basis of the feature fusion model based on fine-grained semantic reasoning,the cross-modal deep semantic association of educational big data is mined from multiple perspectives through the dual confrontation mechanism of global,local and crossmodal multiple comparison and modal interaction.The experimental results show that this method not only has excellent performance on three large public data sets,but also has excellent cross-modal retrieval performance on educational data sets.(3)In the aspect of accurate search of educational big data in crossmodal semantic space,a cross-modal retrieval model based on contextual semantic extension is proposed to solve the problem that existing search engines cannot accurately grasp user retrieval needs.The model simulates the interaction between seeds and context words based on seed-aware attention,and adopts an attention-aware learning structure for both indicative vocabulary and contextual semantic information.By combining the semantic expansion task and the cross-modal hash learning task,starting from the user’s retrieval needs,different retrieval tasks are completed through cross-modal semantic matching.The experimental results show that,compared with other models,the model further improves the cross-modal retrieval performance and recall indicators on three public cross-modal retrieval datasets and the constructed educational dataset.(4)Combining the cross-modal semantic fusion of educational big data,the cross-modal hash learning of educational big data,and the precise search of educational big data in the cross-media semantic space,the cross-modal semantic fusion and precise search of educational big data are designed and realized.system.The three functional modules of the system are as follows:the cross-modal semantic fusion module of educational big data,the cross-modal hash learning module of educational big data,and the precise search module of educational big data in the cross-media semantic space.The function of cross-modal graphic data acquisition and semantic fusion of cross-modal features in the cross-modal semantic fusion module of educational big data is realized.It realizes the function of deep semantic association between different modal data in the cross-modal hash learning module of educational big data.It realizes the function of establishing semantic extension for user requirements in the accurate search module of education big data in the cross-media semantic space and displaying search results based on different retrieval tasks.This thesis realizes the cross-modal semantic fusion of educational big data,the cross-modal hash learning of educational big data and the precise search of educational big data in cross-media semantic space,and designs and develops the cross-modal semantic fusion of educational big data and the precise search system demonstrates the effectiveness and correctness of the system through rich and comprehensive experiments and test results.The algorithm interface is convenient to expand,the user interface realizes friendly interaction,and the operation results of each main functional module of the system are showed in detail.