| Jingdong mall is China’s largest B2C electronic business platform, there are hundreds of millions of traffic visiting every. Jingdong mall has variety of ways to get access flow, of which SEM (Engine Marketing Search) is one of the most important source.SEM is the search engine marketing. This is a way of promotion when using search engine to delivery message to the target user by input different keywords. Through this way the promotion page will get higher frequency and probability to been shown in front of the user, enables users to find and click through to the corresponding shown message to access into the page that the plateform hope to.Jingdong mall has a large number of commodity classification and specific commodity page information. Each page contain different categories of goods. Every different goods has different promotion requierments. Baidu, Sogou,360, Shenma is the majority search enging service supporter in china and share most part of the marketing of china Search einging service. Every day the SEMer has to maintain a lot of account that from different search eiging service supporter. Each promotion account conrresponding different categories, keywords information. To inporve the click rate and impression rate the SEMer must to analyze the data of each accounts that in the syetem of Service supporter, and make real-time update of ratio and prices for keywords. The business logic is complex, the data dimension is much and the data quantity is huge.The system maintenance a lot of accounts that from each search engine service supporter. We fetch running effect data of each account through different platforms, and storage them analysis them and output anakysis report. The system is also providding a unified running operation which effectively improve the effect and operation efficiency.System is mainly to provide the report data, and real-time delivery function. System is based on Hadoop, hive, pig, oozie, Druid, ElasticSearch, node.js, shell, Python and other technology to achieve the data pulls, aggregation, query, modification and service supporter communication functions.The system is mainly divided into fetcher, druid_data_generator, olap_data_loader, UI, save_es, query_es, upload several modules. Fetcher module is mainly responsible for calling the API from each search engine provides in a unified way, and schedule the task. After get the data formatting then and storage into the hive; Druid data generator module is responsible for aggregation the pulled data with the data that store in the background system of JingDng by each dimension; OLAP data loader is responsible for uploading the aggregated data from hive to Druid; UI responsible from Druid query each dimension of the data and display them; save es for saving and edition of data that for SEM; uploader is for uploading the data into the search enging service supporter that saved and edited by save es, and let them updated in real time.I participated in the whole process of project including system requirement acquisition, system designing and development. I write all the code of fetcher, and part of save_es,druid_data_loader. The system is running stabale as expected. The system effectively improved the SEM advertising effect, and enhanced the operation efficiency of every SEMer. |
PDF Full Text Request