A Cloud Computing-based Data Mining Platform Architecture Design And Realization

The development of network techniques brings people in a great deal of information, it also greatly increase the difficulty to find useful knowledge from mass data.The efforts to solve this problem promote the emergence and rapid development of data mining techniques.At present,the data mining technologies and tools have been used in the financial,medical,military,and many other areas of commercial decision-making analysis.Cloud computing is a style of computing in which dynamically scalable and often virtualized resources are provided as a service over the Internet.Cloud computing platform can be used to development high capability programmes.However,it does not provide data reduction service which is one of the bases of data-mining.So the method to implement data-mining system on cloud computing platform has not been worked out yet. To solve these problems,in this paper,a data reduction module for accessing isomeric and new type data is designed and implemented to expand Google App Engine cloud platform, further more,a new data-mining system built on top of cloud computing services is designed and implemented to verify the validity of data reduction module and the efficiency of cloud based data-mining process.Data reduction module supports uniform definition of data sets by using meta-data definition,data set definition and data set instance definition to abstract data type,data structure,location information and so on.Layered thinking model is used and a new muti-layered plugin architecture is induced to enhance the scalability of the system.All system interfaces are RESTful which could be embeded in other applications.Meanwhile,many important thoughts and methods to design and implement the platform are induced in this paper and a prototype of data-mining platform based on this architecture is introduced at last section of this paper.Experimental results show that the proposed method is very promising.