Research And Implementation Of User Log Collection And Processing Algorithm In Cloud Environment

With the continuous development of cloud computing technology and the rapid deployment of cloud platforms,cloud services have gradually been integrated into people's daily lives.Users can purchase or rent storage and computing resources of cloud services to assist their work.However,the security issues in cloud environment are hindering the rapid development of cloud computing.The cloud service providers can guarantee the security of the cloud platform as much as possible in terms of hardware resources and data transmission encryption but they do not monitor users' operations properly.Due to the complexity and diversity of the cloud environment,the traditional log analysis method is difficult to meet the security requirements of the cloud environment.Therefore,aiming at user behavior in cloud environment,this paper builds a Flume-based log real-time and integrity collection platform.To analyze the safety of user behavior accurately,the neural network-based log analysis algorithm is proposed.The main research contents and innovations of this thesis are as follows.(1)Construction of the Flume-based log collection platform.According to the characteristics of the user behavior log in the cloud environment,a custom collection module is designed.A classification-based queue management algorithm is proposed for the integrity of log collection,making full use of the data in the transmission process.Based on the data,the classification mechanism can achieve load balancing so that the buffer queue in the collection platform can coordinate the acquisition rate and the transmission rate in time.For the real-time problem of log collection,the interceptor is used to filter the collected logs in real time.And the log is collected asynchronously with a simple output format.The acquisition system can improve the efficiency of the collection as much as possible while ensuring the integrity of the log collection.(2)In the cloud environment,the traditional log analysis algorithm cannot unify log features and the association among the features,resulting in some important log information being ignored.At the same time,the implementation of the framework does not support distributed computing,which lowers down the analysis efficiency.Therefore,the Detm algorithm is proposed,which uses the dense convolution layer to extract features from the log data.After extracting the abstract high-dimensional features,the memory network is used to analyze the correlation between different types of logs。 Finally,the logs are merged through the fully connected layer then the output is got.The algorithm not only considers the correlation between different types of logs,but also uses distributed computing to achieve high accuracy and scalability.(3)Design and implementation of the behavior analysis system.The log collection module and log analysis module are carefully designed,and the workflow of each module is elaborated.Using the behavior analysis system to conduct experiments on users in the cloud environment,the experimental results show that the prototype system can effectively collect and back up logs and also make judgments on the security of user behavior at the same time,which achieves timely analysis and ensures the safety of the cloud environment.