Design And Implementation Of Anomaly Learning And Detecting System

Today,with the development of information technology,computers and the Internet have brought great impetus to all walks of life.Meanwhile,criminals are increasingly committing crimes by means of computers and the Internet.Attacks from the Internet and threats from the inside of the network emerge one after another.At present,most of the existing anomaly detection systems emphasize the using of algorithm models in the process of classification and prediction,but pay less attention to the adaptation between different data sets and algorithm models,and the influence of hyperparameters of algorithm model.There is also a lack of consideration on the cost of anomaly processing.In order to solve the above problems,this thesis first proposes a method of adjusting the decision threshold of algorithm model.Specially,the mothod is sample-dependent,which extracts the cost information from the data sample.The goal of the method is reducing the average processing cost of the data sample,and the effectiveness of the method is verified through experiments.Based on the decision threshold adjustment method,this thesis puts forward the method which optimizes the configuration of anomaly intervention,in order to guide the system users to process the anomaly as efficiently as possible under limited system resources.This thesis ulteriorly implements an integral anomaly learning and anomaly detection system,which provides functions including anomaly learning,detection and processing.System users can carry out anomaly detection under the premise of the anomaly learning results,and process the perceived abnormal data samples.In the process of anomaly learning,the system can generate recommendatory model hyperparameters for users,so as to improve the accuracy of anomaly detection.With regard to the system architecture design and function implementation,generally the system is based on the Dubbo framework and divided into multiple subsystems,each subsystem is deployed with different system services.All the system modules are highly decoupled which makes the system has productive scalability.In addition,through friendly human-computer interactive interface,the system can visualize the pivotal data,so that system users can control the execution process and results of the system functions better,and can carry out further anomaly processing and analysis easier.The drafted ideas and logic of this thesis is structured as follows:first,this thesis introduces the machine learning methods used in the system,as well as the technology stack used for the development of web application.Then,this thesis analyzes the functional requirements and non-functional requirements of the system.Next,this thesis introduces the method which adjusts the decision threshold of the machine learning model in a cost-sensitive and sample-dependent way,including giving the prototype and instantiated definitions of the cost models,explaining the process of optimizing the threshold with hyperopt and the optimization results.After that,this thesis explains the specific architecture design of the system,the invocation logic among system services,the detailed system deployment mode and the implementation details of critical system services including hyperparameters recommendation service,anomaly intervention and optimization configuration service.Finally,this thesis expounds the functional test and performance test of the system.The test results show that the implementation of each system function meets the functional requirements,and the start-up time and deployment time of system and the execution time of critical services meet the expectations.