Research On Log Analysis System Of IT Operation And Maintenance Big Data Platform

With the continuous expansion of the enterprise IT architecture, the growing number of servers and storage devices, the network also becomes more complicated, which brings great challenges to the IT operation and maintenance. Construction of it operation and maintenance of large data platform is the hot spots research on IT operation and maintenance management. Massive log data for IT operations generated during the processing and analysis are more difficult challenge. The analysis and research on the log of IT operation and maintenance big data platform can improve the response speed of business failures and enhance the competitiveness of the enterprise.Taking Changde Cigarette Factory big data platform as an example, this paper researches on log analysis system in IT operation and maintenance of big data platform. The main tasks of this paper are as follows.(1) Based on big data platform splunk, this paper researchs on log preprocessing to convert massive unstructured log data into structured data. It includes log collection, log cleaning and user identification and then provides the implementation plan.(2)Log traffic prediction is the core of the entire log analysis system. Using time series analysis method to analyze the log, this paper proposes the log traffic prediction model based on Kalman Filter and carries on the experiment analysis.(3) The overall implementation of log analysis system is finished. The server monitoring function, log search center and real time network traffic prediction functions are detailly designed and implemented.