Research On Real-time Bidding Algorithms In Online Advertising

With the development of the Internet,displaying advertisements to Internet users online through the Internet has become the main advertising delivery method.Real time biding(RTB)is an advertising delivery mechanism that has been widely adopted in online display advertising in recent years,it allows that multiple advertisers to bid in real time for the ad impressions generated by each user’s visit.In RTB,the bidding behavior of advertisers is fully represented by the Demand Side Platform(DSP),and the DSP needs to formulate a bidding strategy for the advertiser to use the strategy to help the advertiser automatically calculate the bid during the bidding process.The bidding strategy will affect or even determine the marketing revenue of advertisers.Generally,budget is an important resource limited in auctions.Therefore,how to ingeniously design a bidding strategy to maximize the total value(for example,clicks)of winning display opportunities under budget constraints is a problem that needs to be solved by DSP.This article is to study this problem.Most bidding strategies proposed in the past have a "static" feature,that is,their bidding function during the launch of a new ad is only related to the predicted click-through rate(p CTR)of a single impression.All parameters except the predicted click-through rate of the impression opportunity have been determined based on the training set.All parameters except the predicted click-through rate of the impression opportunity have been determined based on the training set.Therefore,when the real-time bidding environment undergoes major changes(for example,different market competition),these static bidding strategies usually cannot adapt well to the new advertising period.In order to solve this problem,this paper uses a reinforcement learning(RL)framework to learn the best bidding strategy for a highly dynamic RTB environment.In this RL-based bidding strategy optimization framework,the advertising bidding process is modeled as episodic In the Markov process,each plot(usually a day)is divided into a fixed number of time steps,and the bid for each ad display is determined by its estimated click-through rate and bidding factor.The bidding agent will perform corresponding actions to adjust the bidding factors at each time step according to the current state(indicated by the auction information),so that the bidding strategy can adapt to the highly dynamic auction environment.The goal of the bidding agent is to learn the best bidding factor adjustment strategy.This paper uses the Policy Optimization with Penalized Point Probability Distance(POP3D)algorithm to learn the best bidding factor adjustment strategy.Finally,an experiment is compared with several representative benchmark methods on real data sets,and the results prove the effectiveness and advancement of the method proposed in this paper.