The Design And Implementation Of Statistics And Analysis Of The Kepler Union Advertisement Data Platform

After more than 20 years of development,internet advertisement has made remarkable achievements.October 9,2015,the Meituan and the Dianping officially merged and set up a new company,named XinMeiDa.The current,in the 020 field,the Meituan and Dianping has become the largest online platform and scene entrance,including buying,take-away,advertising,leisure entertainment,parenting,marriage,beauty and so on.It covers almost all life services scene,which contains all kinds of data:cross-industry category data,transaction payment data,users browse the data.Union advertising business unit is a sub-sector of Meituan and Dianping's advertising department,which position the external media,for the Meituan group of merchants purchasing traffic,to provide promotional services.Based on this,Union advertising requires a set of advertising data statistics and analysis platform for the department,which could show the data in the form of data advertising,it can be multi-dimensional off-line statistics and real-time monitoring for the follow-up advertising refinement of the delivery,orientation,decision-making to provide data based on the same time,and it can to avoid and solve potential problems and risks in time.This paper introduces the research and development of Kepler Union's advertising data statistics and analysis platform.The offline data model uses Hive,Shell and Python to handle the task scheduling through internal scheduling.The real-time data model uses the combination of Storm and Kafka.The database adopts MySQL and Redis,the business framework uses Spring,Spring MVC,Mybatis.Hive could deal with off-line data.Compared to Hadoop,Hive do not have to write complex MapReduce programming,more attention to HQL,and it reduces the difficulty of development and cost.While real-time data processing uses Storm and Kafka,because it is well suited for adlog processing.The final project is running and on the line,it is of great significace to the operation of the platform and advertising real-time bidding.