Research And Implementation Of The User Behavior Auditing Technology Based On Log Mining

In the field of information security,it is customary to focus on preventing and resisting external attacks,while ignoring the internal threats.However,in recent years,the leaks and the network crashes,which shocked the whole world,were all caused by internal threats.Even if a lot of resources are invested to maintain the border defense,there is no defense against the potential insiders.As more and more security practitioners become aware of the dangers of the internal threats,research on insider threat protection(ITP)has become a trend.Existing threat detection technologies are diverse,but there is still no uniform standard.The most effective ITP is to use log resources for behavior auditing.Based on this background,the paper proposes a research on user behavior auditing technology based on log mining.The main work includes the follow aspects:(1)Behavior acquisition.In open source attack detection dataset,CERT-IT dataset simulates some typical threat behavior data.However,it involves fewer domains of behavior,which is deviated from the real behavior in enterprise environment.Therefore,a Logstash-based collector is designed to extract user's original logs in enterprise environment by path matching,regular parsing and other strategies.The logs will be de-duplicated by HashSet and codes generated by MD5 algorithm.And the logs will be cleaned and fused according to the rules.The integrated data set is combined with the CERT-IT and enterprise logs.And a scheme of extracting user behavior elements from log set and quantifying behavior sequences are designed to provide data support for subsequent threat detection.(2)Threat perception.According to the similarity of user behavior changing with time,a hidden Markov model with time series analysis ability is proposed to model and analyze user behavior sequence.Due to user behavior will evolve with skills and other factors,it is easy for the model to misjudge the complementary data.To solve the problem of data imbalance caused by the sparse supplementary behavior in data set,an improved model is proposed to re-segment the input behavior sequence.In view of the main factors that affect the accuracy of threat detection,a density-based clustering algorithm is proposed to calculate the behavior clusters for optimizing the number of hidden states.And the particle swarm optimization algorithm is used to optimize the initial parameters of the model.Then establish the DP-IHMM model to describe the user behavior.(3)Audit implementation.Based on the optimized DP-IHMM audit model,a behavioral audit system is designed and implemented.According to the fluctuation characteristics of user behavior,forward algorithm is used to calculate the sequence observation probability LGP.And the similarity SIM between the state transition path predicted by Viterbi algorithm and the transition path in the previous period is obtained.abnormal behavior sequence is perceived by threshold comparison.Then the abnormal sequence is extracted to trace the behavior log.The internal threat type is identified by rule analysis strategy,and visualized in the form of chart.The main function modules and the performance of auditing method are also tested.