Design And Implementation Of Anomaly Detection System For IaaS

In recent years,IaaS cloud platform is being adopted by more and more enterprises.However,there are many components and nodes in the IaaS cloud platform.Its operation and maintenance often require a large number of professionals.With the development of anomaly detection,automatic anomaly detection,alarm and early warning could be performed by the system without manual intervention.Providing early warning capability is one of the current trends of cloud application development to reduce the difficulty and workload of operation and maintenance of IaaS cloud platform.Based on the above research background,the anomaly detection system for IaaS is designed and implemented in this thesis.The anomaly detection for IaaS realizes the anomaly detection,alarm and display functions of IaaS by collecting log data and KPl,analyzing and storing them.In order to achieve the above functions,the analysis of log anomaly detection for IaaS is studied.For semi-structured log data,a context-based high-dimensional word vector template algorithm is proposed to extract log parameters.According to the call chain among templates,the data is converted into the fault graph.Convolution neural network(CNN)is introduced to convert the original anomaly detection into the binary classification problem for accurate detection of anomalies.At the same time,the analysis of key performance indicators(KPI)data is studied in this thesis,and the early warning scenario based on mechanical hard disk failures is proposed.The mechanical hard disk anomaly detection algorithm for time-series imbalanced dataset uses a fault sample expansion method based on the generative antagonism neural network(GAN)to handle the case of too few fault samples,and then uses the improved long short-term memory network(LSTM)to classify the original data for early warning failure.It is able to monitor all the mechanical hard disk data of the platform,predict hard disk failures in a timely manner,and ensure the availability of data on IaaS cloud platform.In this thesis,the research background of the anomaly detection system for IaaS is introduced.Then the related technologies are investigeated,and the requirements of the anomaly detection system for IaaS are analyzed.Next,the related algorithms based on IaaS cloud platform logs and KPI for anomaly detection are proposed.Moreover,the overall architecture of the anomaly detection system for IaaS is illustrated together with the detailed module design.In the end,a series of tests are conducted to verify the effectiveness of the system.