Research On Key Technologies Of Dependability Oriented Anomaly Detection Of Virtual Machines Under Cloud Environment

Currently, Cloud computing has become a mainstream mode of computing and service. In large-scale and high dynamic Cloud platforms, the frequent occurrence of accidents has seriously affected the reliability and availability of Cloud platforms, thus lowering their dependability. This thesis aims at detecting anomalous VMs(Virtual Machines) in Cloud platforms. However, it faces many challenges to accurately detecting anomalous VMs, e.g., the huge number of VMs, the large number of collected performance metrics, and real-time monitoring requirements.This thesis studies several key technologies in detecting anomalous VMs in large-scale Cloud platforms. It designs a framework for detecting anomalous VMs. It proposes an unsupervised feature extraction algorithm and a supervised one for unlabelled and labelled datasets respectively. To solve the key problems of detecting anomalous VMs in large-scale Cloud platforms, it systematically studies anomaly detection algorithms based on SVM. Finally, it verifies the algorithms through experiments on three kinds of datasets: Cloud platform datasets, synthetic datasets, and standard datasets. The main contributions of this thesis are listed as follows.(1) This thesis introduces the definitions of anomaly and anomaly detection into dependability. It reviews the history of the development of dependable computing. It expounds the connotation of dependability and its five attributes, i.e., availability, reliability, safety, integrity, and maintainability. It clarifies the relationship between the concepts of error, fault, failure, and anomaly. Thus, it provides important foundation for developping solutions in enhancing the dependability of Cloud platforms.(2) This thesis designs a framework for detecting anomalous VMs under Cloud environments. It summarizes the callenges in this area. It describes the function of each module of the framework. It determines the main research contents. For two important concept, i.e., runtime environment attribute set and performance metric set, it gives formal definitions.(3) This thesis systematically studies the unsupervised and supervised feature extraction algorithms for high dimensional sample sets of performance metrics. It formalizes the problem of feature extraction, and summarizes the challenges. It detailedly analyzes and derives the principles of PCA, LDA, UFLDA, and ICA, and points out the insufficiency of these methods. For high dimensional unlabelled sample sets, it proposes an unsupervised feature extraction algorithm based on unsupervised fuzzy linear discriminant analysis with kernel(UFKLDA). For high dimensional labelled sample sets, it proposes a supervised feature extraction algorithm based on supervised ICA with kernel(SKICA).(4) This thesis systematically studies relevant algorithms and strategies to solve the challenges of detecting anomalous VMs in large-scale Cloud platforms. It formalizes the problem of anomaly detection, and summarizes the challenges. For the scenario of dimensionality reduction and simultaneously reserving a small set of original performance metrics of the sample set, this thesis puts forward a feature selection algorithm, i.e., IRFE. For the challenges of multiple anomaly categories, imbalanced training sample sets, the increasing number of training samples, this thesis studies anomaly detection algorithms based on multi-class SVM, imbalanced SVM, and onling learning SVM. It also proposes relevant strategies for anomaly detection.(5) This thesis conducts extensive experiments and analyses on the proposed algorithms. It introduces the performance metric set of VMs. It analyzes the problems in anomaly detection by using incremental performance metrics, so as to determine the data source of anomaly detection, namely the original performance metric data. It collects the datasets of Cloud platform through fault injection. It also adopts synthetic datasets and standard datasets. Based on these three kinds of datasets, this thesis conducts extensive experiments and analyses on the proposed algorithms.In summary, this thesis designs a framework for detecting anomalous VMs under Cloud environment. It puts forward or improves a series of algorithms, so as to solve the key technologies in this area. For each involved concept, it gives strict definitions. In addition, it formalizes each research problems. The experimental results show that the designed anomaly detection framework is reasonable; compared with the existing algorithms, the proposed unsupervised and supervised feature extraction algorithms can effectively deal with non-Gaussian sample data; the extracted features are beneficial to anomaly detection; the designed anomaly detection algoritms can solve the challenges. Therefore, the designed framework, algorithms, and strategies provide a solid foundation for ensuring the dependability of Cloud platforms.