| In recent years, with the rapid development of the Internet advertisement market, many major Internet companies have published their own Internet advertisement trading platform. Billing system is an important and indispensable part of the whole Internet advertisement trading process. According to the actual business requirements, this paper designs and implements an Internet advertisement trading billing system to support the billing requirements of advertisement trading platform.In this paper, java and scala are used as programming language to develop the advertisement billing system, the main contents are composed of advertisement anti-cheating and advertisement billing. The module of advertisement anti-cheating is used for discriminating cheating ads, which is not charged so that it can protect the interests of advertisers. This paper presents rules for judging cheating ads based on statistical method; in order to avoid the judging result being arbitrary, a scoring algorithm is proposed to calculate the probability of advertisement cheating, so that the cheating ads can be filtered smoothly. The module of advertisement billing is implemented based on spark. Spark is an in-memory, scalable and fault-tolerant computing framework. It processes advertisement data in a distributed environment, filters cheating ads, calculates the amount of billing and generates logs. It makes full use of efficient and fault-tolerant features of a distrubited system, providing billing services with scalability and high availability. In order to avoid that the entire job in a distributed system is slowed down by the heavy pressure on a single node while others are idle, this paper proposes a solution to do fragmentation for large-scale data, so that the amount of data in each fragment are within a reasonable range, which means data can be evenly distributed to every single node. In order to solve the performance bottleneck caused by blocking during network access, asynchronous interface is used to improve system performance. Since the data processed in the system is related to money, a system failure will directly lead to accounting losses which should be avoided. In order to minimize losses, avoid risks, the system has carried out a number of monitoring indicators, abnormal circumstances can be timely alarmed.After testing and actual on-line running, it is proved that the system can effectively filter cheating advertisement, processing about one hundred million advertisement data every day, and the designed performance of the system is higher than the average load of online traffic. Short time data spikes can be well handled. During the entire process, the important data indicators have monitoring, and key actions are logged, so that in case of abnormal circumstances troubleshooting will be easy. The system has several good features, high scalability, high availability and fault-tolerant, which can well support the advertisement billing requirements, and has high practical value. |
PDF Full Text Request