Research And Implementation About Dynamic Balance Of Computing Resources In Online Web Mining Base On Cloud Platform

With the rapid development of Internet, the web information is also increasingly diverse. Network media has been recognized as the "fourth media" following the newspapers, radio and television. And it's hard to master the correction and the range of the media information. Traditional online Web mining systems cab detect the public opinion information on the web. And then guidance and supervise the public opinion. However, the amount of web information is different in a different time. The traditional online web mining system does not consider this difference, which affect the system's real-time and also reduce the resources utilization.Cloud computing is a new method of shared infrastructure. The user can get the required resources on-demand via the Web and the resources can easily extend. In this paper, in order to use the advantage of the cloud computing we design an online Web mining system base on cloud platform. And put forward two strategy of dynamic balance of computing resource and the concept of virtual machine pool. This paper aims to improve the real time of online web mining and computational resource utilization. This mainly includes the following three aspects:(1) For traditional online Web mining for Web does not take into the changes in the amount of web information, this paper design and implement an online Web mining system base on cloud platform. And to take full advantage of the characteristics of the cloud we design a web crawler based on the template and the information processing base on computing resources dynamic balance.(2) In the process of information processing, this paper put forward two strategy of dynamic balance of computing resource. The two strategies can make the system balance the computing resource base on the amount of web information the crawler collect. Experiments show that two strategy effectively improve the satisfaction of real-time systems, and also improve the utilization of computing resources.(3) As the changes in the amount of information is not regular and recurring, so in the process of the dynamic balance of computing resources the number of virtual machines changes significantly. And it need a lot of time when releasing and applying the virtual machine. This reduce the system's real-time. For this question this paper put forward the concept of virtual machine pool. Virtual machine pool is responsible for managing of virtual machine releasing and applying. And it allow the system to repeatedly use the existing virtual machine.