Research On Log Analyze System For Job Schedule

With the quick development of network and information technology, information system scalegrow more and more larger, how to monitor the important data of system become a problem must tobe resolved. Computer log as the true feedback of system, it is logging every event and their resultwhen system is running, so analyzing log file can mine some important information hard to find, andprovider a lot of service for user.Log Analyze System as a total solution for log analyze, it is composed of several computers, andresolve the analyze work through network. System is facing mass log, and offers real-time service foruser. Log Analyze System produce lots of analyze jobs per period, and the jobs schedule to thecomputer to run. Log Analyze Job needs a large amount of calculation, real-time response and highaccuracy. So the schedule module is core of Log Analyze System and its capacity determines thesuccess of whole system.This paper analyzes the functions of job schedule module, to make clear schedule module is coreof Log Analyze System, and then summaries some targets to finish an efficient module. Paper designsan improved version job schedule module by job schedule theory and the experience of developing aLog Analyze System for a company. The module is designed for two aspect, architecture andalgorithm. For architecture, this paper uses some central schedule thought and designs an improvedversion of Master-Slave architecture, called Collector-Master-Slave architecture, the new design notonly reduces load of master but also enhances scalability, and then add monitor sub module in Slavefor a new load balancing strategy. For algorithm, paper improves traditional Ant Colony Optimization,and makes the schedule module more efficient for log analyzes jobs and in order to fit log analyze jobfeature add negative pheromone and dynamic start function. In implementation, paper use JAVAprogram language and some other mature frame, such as Zookeeper, MINA and so on. The designedschedule module improves system efficient and reduces hardware cost, has good value of applicationand popularity.