Design And Implementation Of Log Collection And Analysis System For Mobile News

With the rapid development of smart phones and mobile Internet,the traditional news portal business is being challenged by the mobility and convenience.With its mobile,convenient and other advantages,smart phones have become an important mass media,a new platform for media convergence,and more and more mobile phone users have chosen to use mobile news clients to get news in real time.Currently there are a variety of news on the market client APP,fierce competition in the market allows manufacturers to continue to optimize their mobile news business at the same time,also began to try to provide users with personalized news recommendations.The optimization of the business is inseparable from the comparative analysis of the news service data,and the personalized recommendation needs to analyze the user's preference from the behavior of the user using the news client.You can see the development of news clients can not be separated from the support of massive multidimensional data,so it is a very meaningful task to collect the business log of the news client and dig out the valuable information from it.The log collection and analysis system designed for news client design faces many challenges.Considering the data throughput of log collection and analysis system as a whole,the system uses a large number of distributed frameworks to realize the functions of the system,using Flume,Kafka,Storm And HBase,respectively,to achieve the log collection,caching,transmission,analysis and storage functions,and then take into account the log collection and analysis system in the news client business to play an important role,the system also built a complete monitoring module and data display module Business people provide real-time system monitoring and data display services,The solution proposed in this paper is based on Sohu News Mobile APP,and designs and implements a set of log analysis based on distributed framework from the aspects of research background,technology selection and system architecture.It focuses on the log Collection,log distribution,streaming,system monitoring and data display modules.Based on the actual development,this paper analyzes the reliability and practicability of the system,and puts forward a feasible mobile news client log collection and analysis scheme,and shows some functional interface of the system.