Design And Implementation Of Log Collection And Analysis System For Music Application

In recent years,with the rapid development of foreign and domestic internet and mobile devices,the applications based on the Android operating system and IOS operating system are increasing.These applications are involving in the every aspects of life,from the payment of water and electricity charges to the learning,entertainment and listening to music.Internet companies and non-traditional Internet companies are launching mobile applications corresponding to their business to attract users and earn traffic.In mobile application fields,Mobile application developers should not only take note of the attributes of mobile applications,but also take note of the value of the generated data and behavior log by users.These logs not only have a large quantity,but also have a lot of information.The key issue in log statistics is how to count and query the effective and intuitive data from these large amounts of data.This paper mainly introduces the background and significance of the project,the selection of related technologies and the requirement analysis,outline design,detailed design and implementation,as well as testing of the log collection and analysis system for music mobile applications.The system described in this paper applies lots of distributed frameworks to achieve partial functions.The system mainly comprises log collection,log analysis,early warning system and Spring Boot,etc.The collection of logs includes the gathering and transmission of logs,and the gathering of logs includes the gathering of historical logs and the gathering of real-time logs.The design of this paper uses Flume framework to gather logs and Kafka message queue to transmit logs.Spark computing engine is used to analyze logs,and the part of results of analysis are stored in HBase.Finally,the background system processes business requests and returns data to the front end.The front-end is Html static page,including login modification information,crash log processing,statistical log viewing and other functions.This paper will design and implement the whole system and each module of the log collection and analysis system for music mobile applications.I am mainly responsible for the log collection module,log analysis module,early warning module,system background and I also participate in the development of some front-end modules.The technology related to the system devised in this paper comes from my internship project.Starting from the actual business needs,the system is studied,designed and implemented.The system is able to satisfy the mobile application operators to control and observe the log data,and it can also facilitate developers to view some log data and assist developers to develop applications.In the meantime,it can record some data for the convenience of future activities.The system also has a certain robustness and scalability,and it can cope with the changing business needs.