The Design And Implementation Of Analytic System For Mobile Advertisiong Data Based On Spark

Mobile APP is widely used in recent years,enriching our way of life.Mobile applications generate massive amounts of data every day that contain informations to do with users.Mining data hidden in the user's behavior,as well as taking their convergence and personal preference into consideration,on the one hand can help mobile service providers to improve their business acument.On the other hand,it offers a better experience to users.Therefore,analysis and recommendation for mobile data is of great value for research.Due to the problem of information overload,the scale of data is always too huge for mobile recommendation system to deal with,standalone equipment seems powerless to process it.Recently,Spark platform becomes very popular to handle a complexity computation for big data,it is designed to compute based on memory,along with its own but flexible abstract data structure.Now Spark platform has become one of the hotspots both in industry and scholars.This paper carries out researches on the popular processing technologies for big data in the first place,which includes Flume,Kafka,Druid.io as well as Spark platform;Then it takes an introduction for some traditional recommendation algorithms,and analyzes the similarities and differences between mobile Internet and the traditional one,which provides a theoretical basis for the third module of the whole system;Last but not the least,the paper studies and designs an analytic system based on advertising data of mobile platform by building a distriduted,highly available data transmission and an pre-processing caching mechanism,in the meanwhile providing an online query module that can aggregate data at high speed and an offline processing module based on Spark.The system overrides source APIs of some components,customizes the data channel,and integrates the realtime processiong technology for big data.Also,in the paper we designs the experiment based on the mobile user advertising data to show the functional demonstration through the actual case analysis.