Research On BN-DeepFM Model For CTR Prediction Based On Undersampling Technology

Click-through rate prediction,as a core issue in the field of advertising,has received continuous attention from scholars in recent years.However,the following problems still exist in the prediction: First,the amount of data is large.In the context of the large scale and rapid growth of Internet users,the advertising log data can reach one million levels a day.Therefore,higher experimental equipment and higher technology are required by enterprises.How to make full use of equipment and models to improve big data training and prediction speed is one of the difficult points in advertising click-through rate prediction.Second,high-dimensional sparse feature is difficult to deal with.From feature extraction to model training on advertising logs involves the processing of high-dimensional sparse features.For example,there may be thousands of device numbers,then a feature dimension reaches thousands.Moreover,the traditional feature dimensions after encoding can reach a million levels and are very sparse.At the same time,traditional machine learning models that predict click-through rates often require expert feature processing,such as feature combination.Such artificial feature processing often cannot accurately grasp the implicit cross-relationships inside the features,and fitting high-order features requires a lot of effort.Third,the data categories are not balanced.We consider the problem of advertising click-through rate estimation as a binary classification model,with1 representing being clicked and 0 representing not being clicked.Individuals often click very few ads on the Internet when they are pushed.Maybe we only click one or two of ads on a page,and there will be an imbalance in training data.Existing research shows that most models are trained on a relatively balanced sample distribution,and extremely uneven advertising data may affect the overall estimation effect.The batch normalization optimization method is an effective neural network optimization method proposed in 2015.At present,it has not been used in the problem of ad click rate estimation.In this paper,based on the research of existing models,the pre-processing of the data using resampling technology greatly improves the model training speed.At the same time,adding a batch normalization layer to the DeepFM model structure makes the model prediction performance significantly improved.This paper proposes an improved model for advertising click-through rate: a model for estimating the click-through rate of BN-DeepFM ads based on undersampling technology.The specific contents include: First,in order to eliminate the impact ofcategory imbalance on the prediction performance of the model,the undersampling technology is applied to the ad click rate data set.The result shows that the prediction performance of the model is almost the same as that before the sampling,but the model training speed is greatly improved.Second,the effects of different activation function layers and batch normalization layers on the optimization degree of deep neural networks are studied.The results show that the relu activation function is better than the sigmoid activation function and the softmax activation function,and adding batch normalization layers to the hidden layer has better results than using the relu activation function.In theory,the batch normalization layer method can improve the generalization ability and training speed of the model.The experimental results show that the prediction performance of the model is improved by 1% compared with the batch normalization layer,but the training speed of the model is almost not improved in the experimental results of this article.The reason may be that the amount of data decreases after the undersampling process,and the model's The training speed has been greatly improved compared to before undersampling,so the batch normalization of the data set on the undersampling technology has not played an acceleration role.Third,the embedded layer is processed for high-dimensional sparse input data.Feature vectors with different dimensions after one-hot coding are mapped into low-dimensional dense embedding vectors with the same dimensions,and the mapping weights are obtained by training using a factor decomposition machine model.Fourth,the deep learning model is compared with the shallow learning model on the data set,and the results show that the deep learning model has better prediction performance.This is because shallow learning models usually only use low-order features,while deep learning models can automatically learn high-order features,which can make better use of the information behind user behavior.