Research On Deep Learning Algorithm For CTR Prediction

With the rapid development of the internet and big data technology,advertising industry has undergone dramatic transformation,gradually shifting from traditional radio and TV advertising to online advertising that can achieve precise delivery.Click-through rate(CTR)prediction plays an important role in accurating delivery of online advertising,which is highly closed to the satisfaction of user’s experience and economic benefits of advertisers.It has therefore attracted great attention from researchers in this filed.At present,CTR prediction model based on deep learning has become the mainstream trend of this research area.Although great results have been achieved,there are still some problems need to solve:(1)The deep models usually utilize three-or four-layers of feedforward neural network to explore high-order interactions,but with the increase of layers,problems such as gradient disappearance and degradation will occur,as a result in the limitations of the improvement of model performance.(2)User’s interest is the key to improve the performance of CTR model.The present models usually use gated neural networks such as LSTM and GRU to extract user’s interest hidden in the historical behavior sequences.However,these networks have sequence dependencies and cannot be processed in parallel,resulting in low training efficiency.(3)User’s interest will shift with the changes of internal and external factors,while these changes will directly affect the subsequent behaviors of users.However,the existing deep models seldom consider the impact of the change of user’s interest on CTR.For problem(1),this work utilize the mixed product interaction of inner product and hadamard product to explore low-order interactions.Gating mechanism is employed in the residual network to construct dense gated residual network and skip gated residual network,which alleviates the problems existing in the feedforward neural network.On this basis,the corresponding CTR prediction model was established.In order to further enhance the model’s ability to explore feature interactions,this thesis adaptively fuses the two gated residual network models with a gated strategy to obtain a hybrid model based on the gated residual network.Experimental results demonstrate that our model significantly outperforms state-of-the-art baselines.For problems(2)and(3),this work proposed a hierarchical attentive behavior-based network model.This model extracts potential user’s interest from rich historical behavior sequences by constructing a Transformer with auxiliary loss supervision,which not only improves the training efficiency of model,but also obtains more effective information of user’s interest.On this basis,this work combined the local activation characteristics of the attention mechanism and the sequence modeling characteristics of the minimal gated unit(MGU)to construct a minimal gated unit with attention input(AIMGU),which activates the candidate advertisements relevant interest information and model the evolution process of user interest to obtain more accurate information of user’s interest.Finally,we stitched user’s interest features with the features of advertisements and fed into dense gated residual network and jump gated residual network,respectively,to further explore feature interactions.Numerical experiments on the Amazon dataset verify the superiority of the proposed model.