The Design And Implementation Of General Logging System Under Distributed Computing Environment

Logging becomes much more important in today’s internet service. During whenever developing or serving, logging can perform a big role. Software developers use logging to inspect status of system, then find and locate system bug, while bussiness operator gather stats of business, user number, user growth from logging data. As the exposive growth of internet and the size of user, one server won’t be able to server well due to the limit of its resource. Distrubuted computing stands out and becomes the leading trend. But here comes the problem: how to collect logging data with ease and safe while data is ditributed in different servers, difficult to collect and growing fast? This need some serious consideration.In this paper, the functional requirements of general logging system under distributed computing environment, including logging, searching, managing, has been analized, as well as the requirements of performance and safety. All works including ① Building of general logging system under distributed computing environment. Client get server information from configure center. When local data is collected, client send it to server for further process. ② Implementation of high performance server. Server’s job is to save logging data into database. Considering the delay of quering database and performance of server, server has been divided into two parts to increase concurrency: log_if and logd. Log_if collect logging data from client, read, validate and save into share memory while logd execute the query. As to the database, system choose MySQL because of its user-friendly interface and good performance with the enhancement of using infobright as its saving engine. ③ Design and implementation of general logging system. With the protocal of interaction between API and agent, API can be implemented with different language. User using python, java and Javascript can also save logging data in this system. ④ Design and implementation of high performance client. Client has two parts: poor API and rich Agent. The good practice for API is that the less API does, the better. API returns immediately after saving logging data to share memory, nothing more. Other job has been left to the agent: saving local log, sending logging data to server. ⑤ The implementation of query system. Query system is a web system. With the help of Bootstrap, jQuery and various frontend plugins, query client offers a user-friendly, easy-to-use user interface. Also, while developing this system, in order to meet user’s habits, much effort has been made and their opinions have been taken into account.With all functional test and performance test has been applied, this whole system has been verified to meet all users need and make minimal performance impact. This whole system had been tested. All feature had been verified and the performance is well.