Research Of Safety Monitoring Technology Of Cloud Computing Platform Oriented Hadoop

Cloud computing represents the development trend in the IT field turns to intensive,large-scale and specialization, is the profound revolution going through in the IT field. Itimproves the efficiency of the use of all kinds of resources greatly, but also brings greatshocks and challenges to the asset security and privacy protection of user information. As anopen source basis cloud computing framework, Hadoop gets more and more attention in thebusiness community, but the issues of cloud computing platform security has not beeneffectively resolved yet. Therefore, the weakness in Hadoop security mechanisms has becomeone of the main problems which hinder its development. So, the Hadoop platform securitymonitoring technology, to a certain extent, will enhance the users’ trust in Hadoop, thuspromote the development of Hadoop.In this paper, based on the research of Hadoop security issues, aiming at the difference ofHadoop cluster data security and resource security, we proposed a data access anomalydetection technology of user behavior and a resource consumption anomaly detectiontechnology, to monitor the security of Hadoop cluster, thus improve the security of theHadoop cluster. The main work of this paper is as follow:First of all, according to Hadoop HDFS data access mechanism, we propose a useraccess behavior anomaly detection technology based on a hidden Markov model, which isdifferent from the others based on correlation analysis. This method aims at a single user’smicro command sequence, avoids data mining process during correlation analysis andpretreatment process of the multiple feature data, which reduces the time and spacecomplexity greatly, improves the performance of real-time monitoring to user’s abnormalbehavior.Secondly, according to the distributed storage and distributed computing of resourceconsumption characteristic on Hadoop platform, we propose an anomaly detection technologybased on the time series of resource consumption, with the k-nearest neighbor methodapplying to the time series model, combining with the sliding window, realized the subsidiarysequence pattern of local anomaly detection, thus found out patterns of anomaly. This methodimproves the general anomaly detection technology based on burst point, thus find out the slow changes, to achieve the detection of abnormal subsidiary sequence patterns.Through the research, the user data access security will be further protected, while theHadoop cluster resource safety monitoring ability makes certain progress, and provide aneffective reference for the goal of providing reliable service performance to Hadoop users.