Design And Realization Of The Massive User Behavior Analysis System

With the development of Internet industry,convenience brought by Internet is inseparable in people's life.The government's support to Internet + has created the integration of numerous traditional industries and Internet,which not only has changed the development mode of industries,but also has created a large amount of data.People's all behaviors on the Internet are recorded by formative data,which has significant value to analyze user behavior,improve service value as well as predict willingness to buy and the trend of public opinion.In Internet industry,a great amount of behavior log data is created every day,which is calculated with the unit of TB frequently.Therefore,how to collect,process,store and present users' massive behavior data better,quicker and more accurately has become an impending demand.In the field of user behavior,the scientific application of data analysis method and theoretical derivation can reveal the inherent law of user behavior completely.It can help enterprises realize multidimensional cross-over analysis,and help enterprises establish agile business intelligence decisions that respond rapidly and adapt to changes.Utilizing user behavior data can create better service,which is meaningful to any organizations.Ever since its creation,massive user data needs to go through multiple data processing stages to pass cloud service and further provide decision reference values to decision makers,or provide project support of decision-making data to seniors.How to collect efficiently,process and store massive,heteroid,real-time and diversified user data to realize rapid,correct and timely response of large-scale and complicated user data query and pivot has become a meaningful research direction.With the passage of time,the value of user behavior data is gradually being tapped and its importance is also being recognized increasingly widely.The study analyzes the schemes and technologies about the user behavior data analysis system in the industry from a systematic perspective.Besides,based on the advantages and characteristics of currently available emerging technologies,the study proposes a systematic scheme that can satisfy the analysis of massive user behavior data.The scheme involves a complete set of solutions that cover data collection and reception,data processing,data warehouse,data analysis and data visualization.The study mainly consists of the following parts:(1).The study establishes a complete subsystem for data acquisition and reception,including data collection strategy,solutions to a variety of abnormal problems,scalable design of the system etc.(2).The data processing link is constructed,and behavior data collected from different sources are preprocessed and cleaned in the log format to produce data that can be used in the standard calculation.(3).The user behavior data warehouse is constructed,and the processed data are integrated with each other according to relevant needs to develop user behavior data warehouses of different products as the core data of the user behavior data analysis system.(4).A platform is established to analyze the user behavior data.According to the value judgment of user behavior data analysis within the industry,the study provides fast user behavior data analysis templates and data visualization programs including algorithms such as event analysis,funnel analysis,retention analysis,user path etc.In addition,the independent analysis program offered for the users is retained so that users can independently analyze user behavior data and explore the data value.The massive user behavior analysis system which is established according to this design project in this paper has been deployed and used in the largest online taxi-hailing service Internet company in China.Complete open source and self-programming system are adopted in this system,which guarantees transmission reliability of data in each sub-system.Mainstream design ideas and advanced technology are adopted in massive user behavior analysis system to guarantee the integrity and extendibility of the system and provide a simplified,quick and large-scale data analysis product.Therefore,it can simplify analysis process,improve efficiency and reach business directly.From design to being used,the system has been through a year of design and development.At present,it can provide stable,accurate and timely data support to analysts and decision-makers of user behavior analysis.