Grid-based Data Mining Platform Architecture Design And Its Implementation

In recent years, data mining technology has been successfully applied to every walk of life. However, some factors restrict data mining technology for further development, such as: non-trivial property of data mining algorithm choices and parameters setting; the complexity of data mining software installation; data security and privacy issues, etc. Grid computing technology for its handling compute-intensive and data-intensive resources integration capabilities provides an unprecedented space for data mining development, grid is becoming an ideal platform for data mining, that is because grid provides a powerful computing capabilities; grid computing in resources sharing, security mechanism and task management mechanism, makes data mining researchers to focus on the high level knowledge discovery process.Combining the problems of data mining technology and grid computing technology, this paper proposes a grid-based data mining platform BillionGrid. The system architecture is divided into four levels, from the bottom to top are: basic resources layer, including hardware resources, data mining web services and so on; grid middleware layer, we use Globus Toolkit as grid middleware, Globus Toolkit provides basic grid services to support upper layer; core service layer, including user information management, service and data management, client management, visualization management and task management; user interface layer, which provides a web portal for users to use the platform.Secondly, semantic description for Data mining tools can not only make users to transparently access to and effectively integrate from data mining resources, but also to have better understanding of the data mining process and results. So we design a data mining ontology, with which we describe our data mining web services. On this basis, we provide semantic search of web services, which greatly enhances the performance of finding data mining web services.Finally, we design a service recommender system, the system keeps records that how people use algorithms. According to the idea of association algorithm, this paper implements a service recommendation algorithm to offer suitable data mining web services for users, which enhances intelligence of the platform.