Research On KNN Log Processing Based On Storm Improvement

With the Internet,the Internet of Things industry has developed rapidly.Therefore,a large number of access logs are generated every day.If a large amount of logs can be analyzed.Then a amounts of valuable information will be excavated.Currently,it is not only necessary to process massive log information,but also needs to be processed in real time.Enables real-time analysis of results.This article has carried on the thorough research to the big data frame Storm,and has built a data analysis system in combination with the components inside the big data ecology circle Flume,Kafka,Redis,Hbase.This article improves the KNN algorithm.The improved KNN algorithm is used to build a data analysis system.Because the traditional KNN algorithm is sensitive to the density of the sample,the density of the sample seriously affects the classification accuracy of the algorithm.And as the amount of data increases,the execution time of the program increases dramatically.Therefore,this paper improves the classification accuracy and execution time of algorithms based on density-based KNN.Combining the improved algorithm with the big data framework Storm,the distributed implementation of the improved algorithm is completed.This further reduces the execution time of the algorithm.The improved KNN algorithm is suitable for log data analysis in big data scenarios.