Research On Web Mining Service Platform And Data Backup Technology

With the rapid development of Internet technology, the information on the Internetis more and more abundant. To deal with the massive information, it is necessary toadopt new technique to mine and manage Internet information. An effective system isthe basis of providing user-friendly information mining service. For this requirement,our research mainly focuses on the following aspects:1) Design of the framework of web information mining platform2) Functional Design of web information mining service3) Disaster Recovery for the key data of the platformConcerning about the above-mentioned three key-points, we have done the followworks.(1) We proposed the design of platform for web information mining service system.The platform is based on the idea of J2EE light framework design, adopting MVChierarchical model, dividing each modules by its function. We implemented thisweb-mining system based on Solr with high efficiency and usability. This systemprovided a steady environment for information mining task. It satisfied the practicalapplication need in service stability, response speed and throughout rate. Besides, itsupports extensible development well for its feasible architecture.(2) Concerning the real need of web information mining, we researched anautomatic web page classification method based on meta-data. This algorithm canimprovement the traditional classification effects with the help of web meta-data. Thisalgorithm classify web page with high precision in a high speed. For providing preciseweb page recommendation service, we put forward a similarity calculation algorithmbased on weighted meta-data. We used query expansion techniques based on relevantdictionary to reach the goal, pushing information to user. Relevant experiment hasdemonstrated, these methods proposed in this paper can put into practice.(3) Researched the typical data backup and recovery techniques. Considering ourreal need--guaranteeing the integrity and availability of index data, we implementredundant backup for disaster recovery. Among them, we employ the incrementalasynchronous replication technology to achieve full text indexing database backup, andbased on the log playback technology we realize hot-words database backup and recovery.(4) Realized the visualization function for displaying dendrogram for the textclassification, pie-chart for web page clustering result, and line graph of hot spotinformation transition based on JQuery.