Design And Implementation Of Advertising Click Rate Prediction System Based On MapReduce

With the rapid development of the information technology, the big data era has arrived. Relying on big data and the Internet, many traditional be updated or even be reversed. Search advertising has become one of the main source of income of the Internet industry, the operation mode of the advertising is often as keyword bidding, advertisers pay for the cost of these keywords, the main payment method for PPC (Pay Per Click, PPC). Advertisers pay for each click costs; The popularity of advertising is described by the click-through rate(CTR); And the advertising media revenue is CTR * CPC. Thus, the prediction of advertisement clicks is particularly important.Firstly,we use MapReduce framework to deal with massive advertising data, then based on bayesian network structure between advertising keywords similar model, in the next mass of storage on HBase, bayesian network probabilistic inference, and then get to predict advertising clicks. Based on the above ideas advertising clicks prediction system is realized, successfully solve the mass of data in the case of advertisement click rate prediction. In this paper, the main work summarized as follows:1) Data preprocessing.In this thesis, the system is mainly based on MapReduce framework to analyze users’ search log processing, after to extract the valuable data stored in the HBase.2) Construction and storage of large scale Bayesian networks. In this thesis, the system is based on MapReduce distributed computing framework, the advertising keywords as a Bayesian network nodes, it constructs Bayesian network structure of directed acyclic graph, and then based on Bayesian network structure of directed acyclic graph, parallel computing the conditional probability parameters of each node table, and finally to complete construction of Bayesian network with key/value pair< key, value> form parallel storage to HBase table.3) Advertising Click Rate Prediction Based on large scale Bayesian networks. In this thesis, the probabilistic reasoning of Bayesian network is transformed into datzzquery processing on HBase, at the same time based on MapReduce programm in model to achieve large-scale probabilistic inference of Bayesian network, and then predict advertising clicks.4) Based on the research above, we will design corresponding system, consists of three modules:data pre processing module, large scale Bayesian network constructing module and an advertisement click rate forecasting module, and realized based on MapReduce advertising click through rate prediction system.5) At the end of this thesis, based on the real business data,the functional and non functional of the system was tested.