Design And Implementation Of Information Management System Logs Analysis System

The Internet is visited hundreds of millions or even billions time every day,and produce a large number of log files which records user access rules.This files is a very valuable asset.How to deal with these massive log data,has gradually become a hot issue in the industry.Through the hadoop platform,this paper takes the information system log as the starting point.It designs and implements an information system log file analysis system which meets the actual requirements based on the characteristics of information system log file and the business characteristics of log analysis.This paper creatively designs a software architecture that can correctly meet the business requirements of information system log analysis,and expounds the design idea of this architecture,and discusses the reasons for choosing the architecture.In this paper,the system is designed as log collection part,log storage part,log processing part,processing result storage part,the results display part.Combined with the actual business needs,this paper gives the specific software implementation.This paper briefly introduces how to configure the Tomcat log in the sys tem,elaborates the realization principle,the advantages and disadvantages of the Flume a rchitecture,and compares the similarities and differences between Flume and Scribe in t he section of Log data acquisition.In the part of realization of the system log storage,th e HDFS storage principle and the advantages of this selected data storage method are int reduced briefly.In the design and implementation of the system in the log processing pa rt of the realization,it introduces the principle of MapReduce programming model,de scribes the advantages of MapReduce programming model in dealing with large data an d compare the similarities and differences between Hive and traditional relational data.I n the part of the realize the storage of processing results of this system,this paper shows the advantages of MySQL database in achieving this part.In the discussion of the result s of the system in the display part of the realization of a simple configuration to achieve a small information display system,and introduces the visual control ECharts.In the latter part of the article,taking the postgraduate management information system of Shandong University as an example,it shows the concrete realization code and implementation effect of the log analysis system under the architecture.