Design And Research Of Network Information Micro Platform Based On Theme Crawler For High Quality Courses In Colleges And Universities

In recent years,with the development of information technology in Colleges and universities and the increasing speed of the mobile network,the Internet has become the common demand of teachers and students.The construction and popularization of all kinds of excellent course websites facilitate the information transmission between teaching and learning.On the other hand,there are also problems of different websites,such as array,large amount of information and low resource sharing,which are not conducive to achieving unified retrieval.In order to further improve the utilization rate of existing excellent course website resources,this paper will design and research the micro information platform based on topic crawler.This paper systematically analyzes the business processes of excellent courses,network information and micro platforms,and puts forward the operation process,key technical indicators and research status of related systems of excellent courses,network information and micro platforms.Constructing the WEB server through the Tencent through cloud services,load balancing and other means to avoid WEB server downtime caused by frequent access.The paper introduces the working principle of the theme crawler systematically,and designs and studies the workflow of the theme crawler from five aspects:structured data extraction,web structure similarity computation,text denoising,Chinese word segmentation and topic extraction.The basic steps of the web page classification are introduced,and the basic principles of the simple Bias classification algorithm,the TF-IDF algorithm and the cosine similarity are emphatically analyzed.The Naive Bayesian classification algorithm is used to solve the calculation of structural similarity in Web data extraction and processing;calculation of theme words using the TF-IDF algorithm to the whole article on the theme of weight determination of words and index to establish a decisive basis;similarity algorithm to evaluate the similarity between the use of Yu Xianxiang.Finally,according to the actual application scenarios and vertical search engine technology,we can grasp the network resources of excellent courses in universities through the theme crawler,and then use WeChat public platform to build the corresponding curriculum micro platform.In order to realize the centralized management and open sharing of high quality teaching resources,the web curriculum resources,which are originally independent,are collected and integrated through the crawling technology.From the perspective of the management of the curriculum platform,unified entrance and unified certification will provide many conveniences for the teachers and students.