Design And Implementation Of Educational Resource Sharing Platform Based On Hadoop

With the continuous development of Internet technology in the field of education, business data related to the education sector is growing exponentially, showing the characteristics of massive, diverse, complex, heterogeneous and high value. Abundant resources for basic education provide adequate references and educational information for the majority of educators, students and parents. However,due to the data fro m various heterogeneous application systems,and the lack of effective integration standards and means, the distribution of these resources are highly dispersed,the content is complex and disorderly,the standardization degree is low, users are often difficult to quickly and accurately get the information they need. Therefore, research and design education resource sharing platform to achieve efficient mass data storage and management, improve the speed and accuracy of user access to resources is a hot research topic educators.Focusing on the above issues, this paper designs and realizes the educational resource model based on the analysis and study of the characteristics and classification of educational resources. The distributed Hadoop architecture is introduced, focused on the theory and technology of Hadoop Distributed File System(HDFS), distributed computing model(Map Reduce V2),open source database(HBase) and search server(Solr). Build a Hadoop cluster and integrate HBase and Solr C loud for designing and implementing a distributed real- time resource retrieval scheme. The Hadoop technology is used in the educational resource and user log module,as a result, the educational resource sharing platform based on Hadoop is realized.This paper focuses on the distributed storage of educational resources and real-time retrieval technology, the main work includes:(1) This paper analyse the basic principle and architecture of Hadoop platform and pay attention to the HDFS file system, Map Reduce V2 computing model and HBase distributed database. Map Reduce is used in the user log analysis and the combination of HBase and Solr Cloud is used in the distributed storage and real-time search of educational resources.(2) Based on the requirement analysis and use case design, the system goal, the structure model and the functional modules of the educational resource sharing platform based on Hadoop are determined.(3) On the basis of the research on the relevant technology of the storage and retrieval of educational resources based on Hadoop, a J2 EE based educational resource sharing platform is designed, which includes presentation layer, business logic layer, persistence layer and database layer and the function module of each layer is designed and implemented.(4) According to the system design scheme, the hardware and software environment and the distributed cluster of the system are built, and the system's function modules are tested one by one to verify the availability of the system design.(5) Finally, the operating conditions and existing problems of the system are analyzed, and the application effect of the system is evaluated.