Research On Association Rule Mining Algorithm And Its Application In Large Data Environment

With the development of computer technology,the Internet of things,big data and artificial intelligence have been used more and more widely in recent years,The gradual increase of real-time predictions of weather forecasting and geological disasters has eventually generated a large amount of monitoring data.All these facts increase the pressure of data processing and analysis,and the difficulty of extracting valid information for users is also increasing.In view of above problems in the area of big data,this paper proposes an optimization algorithm for association rules mining,and discusses its application of the monitoring and warning for the landslide.Firstly,this paper reviews and outlines the research states of association rule mining algorithms at home and abroad,and systematically combs the relevant basic theories.It mainly involved the association rules,Apriori calculation methods and other aspects of in-depth discussion.Secondly,the overall design of the large data mining system is carried out.The functional modules of the system are described in detail,The experimental platform is built.Thirdly,this paper presents a design and optimization method of the algorithm level,and the optimized calculation method according to the temporary table.This improved algorithm can clean up more complex and invalid transactions,and effectively reduces the number of iterative scanning of the algorithm,thus enhancing the efficiency of the calculation method.Finally,taking the analysis of landslide data as an example,the paper makes a data preprocessing of the original noisy data,and uses the fuzzy logic method to obtain the general rule of the landslide.Then the paper finds the frequent itemsets of the relevant reason of the landslide,filters the candidate set and redefines the types of landslide.In the end,the paper concludes the most closest or the most approximate expression of the landslide,which provides a strong basis for the reduction and prevention of natural disasters,and provides accurate prediction criteria for real-time prediction of landslide geological hazards.