Study On Distributed Web Data Mining Clustering Algorithm Based On Mobile Agent

With the development of network technology and distributed data storing technology, distributed Web data mining technology came into being, more and more focused on by researchers. For the distributed Web data mining problem, the communication cost will increase and the executive efficiency will be low when the scale of site data is huge. So, it is advisable to divide the big system into several small-scale subsystems and hierarchically execute the mining job with the mining models, according to the hierarchical designing idea and strategy of dividing and managing. Mobile Agent technology is a entity which can autonomously move from one host to another in a heterogeneous network and communicate with other Agent and resources. The intelligence and mobility of Mobile Agent technology not only reduce the network load, but improve communication efficiency. Supporting asynchronous autonomous interaction, breakpoint operation and concurrent calculation and being secure and capable of learning, fault-tolerance and intelligent routing, Mobile Agent, applied to distributed Web data mining system, technology has a great research and application prospect.The thesis is divided into three parts. Firstly, the thesis systematically studied and summarized the theories of Mobile Agent technology and distributed Web data mining technology. Secondly, a hierarchically optimized incremental knowledge integration model(HOIKI) for distributed Web data mining is proposed based on optimized incremental knowledge integration model(OIKI) for distributed Web data mining and Mobile Agent technology. Finally, apply the model to distributed clustering problems and proposed a Web clustering mining algorithm based on Mobile Agent. Experiments and analysis show that HOIKI is more flexible, nimbler to be implemented and better to be adapted to distributed environment than OIKI. It can effectively reduce communication cost and is especially suitable for solving the large-scale heterogeneous distributed Web data clustering problem.In this thesis, HOIKI optimized the network translation only with the consideration of the distribution of node cluster. However, the situation is more complicated actually in real world. The thesis has some referential value for further study of improving the whole mining performance by establishing more complicated optimizing index.