Adaptive Hierarchical Web Robot Detection Model Based On Immunologic Mechanism

Web Robot is a kind of automatic execution of the program on the network,the increase of its type and quantity has brought a lot of harm,so Web Robot detection has become an urgent problem to be solved.Identifying Web Robot session from the access log is an effective detection method.The rule-based detection method is accurate for the known Web Robot,but it can not detect unknown and changing Web Robot,and the rule base is difficult to maintain.Pattern analysis and learning analysis methods,for unknown and changing Web Robot,through the statistical or learning access characteristics of the detection,the effect is better,but because Web Robot itself is always changing the behavior of the visit,and statistical or learning algorithm is static,these methods in the adaptability and dynamic performance is not good enough.Biological immune system is a multi-level,self-learning defense system,it can dynamically adapt to changes in the external environment,to maintain the body immune self-stability.The computer immune system is established by the principle and characteristics of the biological immune system.The algorithm of clonal selection is widely used to solve the problem of optimization and classification.In this paper,a hierarchical,adaptive Web Robot detection model is proposed to solve the problem of automatic maintenance of known Web Robot rule base and adaptive dynamic detection of unknown Web Robot by using a variety of immune mechanisms,using clonal selection and dynamic clonal selection algorithm.The main work is as follows:Firstly,summarize the status of Web Robot detection and computer immune system research.This paper analyzes the advantages and disadvantages of various Web Robot detection methods,and points out the key issues that need to be solved of Web Robot detection:accurately detecting known Web Robot and dynamically detecting unknown Web Robot.Analyze the key mechanism of the biological immune system and the characteristics of the computer immune system,and explain the reasons for solving the Web Robot detection problem by referring to the immune mechanism.Secondly,an adaptive hierarchical detection model of Web Robot based on immune mechanism is proposed.The model consists of a rule-based detection layer and a learning analysis detection layer,and the supervisory feedback mechanism acts on both layers.In the rule-based detection layer,a known Web Robot is detected by maintaining a rule base.In the learning layer,we first improve the clonal selection algorithm,introducing the penalty factor in the affinity calculation,to optimize the combination of the initial feature set;then improve the dynamic clone selection algorithm,that is increasing the source of the immature detector,performing clonal and mutation on the mature detector,performing the receptor editing on the failure detector,then generate a detector with diversity to detect the unknown and changing Web Robot.The supervisory feedback mechanism works to automatically update the rule base by observing the number of times of the same Web Robot,and dynamically update the detector set by analyzing the detected rate of change of the detectors.Finally,this paper takes the undergraduate educational management system of Wuhan University as the application environment to validates the validity of the key parts of the model,including the feasibility of updating the rule base based on the accuracy of the rule detection layer,the necessity of the feature combination optimization and the performance of detectors.The experiment results show the advantages in accuracy and adaptability of this model.