IoT Platform Anomaly Detection System And Method Based On Log Analysis

As a strategic emerging industry,the Internet of Things is widely used in industrial control,smart cities,military defense and other fields.Massive node differences,heterogeneous network interconnection,multi-source data aggregation,and cross-domain service sharing make the Internet of Things more vulnerable to attacks such as Do S,SSH brute force,XSS/SQL injection,and node capture forgery.System anomaly detection and real-time response become the basis for protecting the security of the Internet of Things system.Massive log analysis is the main means of anomaly detection and judgment.Because the network management system(NMS)needs specific domain knowledge and can not identify detailed network events,and can not meet the basic requirements of Io T security,this paper studies the association method based log template extraction method and autonomous learning based anomaly detection method,lifting object The accuracy of abnormal detection of networked systems,the specific research content is as follows:Aiming at the complexity of the massive multi-source heterogeneous log structure of the Internet of Things,an effective information extraction model is proposed,and the extraction rule mechanism is designed to realize the automatic analysis and cleaning of massive logs.Aiming at the difference of the probability of template words and parameter subwords in the log text,this paper proposes a template word extraction method based on DBSCAN,which uses density to effectively distinguish template words.Aiming at the multiple traces generated by a network event in multi-source heterogeneous logs,according to the association relationship,a template extraction method is proposed to form a log template set of the attack type index.Aiming at the endless emergence of attack methods and the fact that some vendors do not publish the template library,an online template set update method based on similarity is proposed.Experiments show that the average accuracy of the template extraction model based on the above method can reach 98%,which can achieve accurate extraction of massive unstructured compounded network log templates.Aiming at the demand of active anomaly detection of Io T platform,it is necessary to actively learn and classify unknown network events.Based on the unsupervised idea of K-Means clustering algorithm,a method of actively learning network events is proposed,and KMeans-based data is designed.The classification method realizes the conversion to the network events of massive unstructured compound networking logs.Experiments show that the method can achieve event recognition accuracy of more than 95%,and can handle multisource heterogeneous log types with dynamic adaptability.Based on the researched technologies and methods,based on the self-developed microobject sharing platform,ELK-based log collection,analysis,transmission,storage and display systems are developed to detect known attacks in real time,identify unknown attacks,and improve The security of the Io T platform verifies the effectiveness of the proposed method.