Implementation And Application Of The Algorithm Of Mining Association Rules For Stream Data Based On Storm

Finding association rules in massive data is always a hot topic in data mining.Even though many association rules mining algorithms have been developed,new association rules mining algorithms for stream data are still required due to the stream data is continuous,unbounded,generated at a rapid rate,and the knowledge embedded in a stream data is more likely to be changed as time goes by.In order to have a better analysis of stream data,extract association rules from data at multiple time granularities,and improve the efficiency of mining association rules from stream data,this thesis develops an association rules mining algorithms for stream data,named as DPFP-stream(Distributed Parallel Frequent Patterns Mining Algorithm for Stream Data).It is based on FP-Stream algorithm and the parallel programming model of MapReduce.Furthermore,the thesis implements DPFP-stream algorithm on the Storm platform.In the implementation scheme,stream data is emitted by middleware server called Kafka;middle results are stored in Redis through serialization.Multiaspect experimental results and analysis show that the designed algorithm has highly efficiency and stable performance when finding recent association rules from a high-speed data stream,and supports multiple time granularities.This thesis applies DPFP-Stream algorithm to the recommendation system of clothing shopping so as to test the practicability of the algorithm.Through the analysis the rules of trendsetters' and the shopping history of customers,the algorithm real-time updates the association rules,and the system recommends the matched clothes when customers are watching some clothes.Through the accuracy test and performance test,it can be found that the recommendation system responses quickly,so that the experience of the system is perfect.As a conclusion,the DPFP-Stream algorithm based on Storm platform which provides accurate and efficient recommendation can be well applied to the clothing shopping system.