A Research For Click-Through Rate Prediction In Display Advertising Based On Feature Combination

As more and more revenue machine learning has brought for Internet companies in computing advertising,the research on the click-through rate prediction has also been deeper in recent years.In particular,the process of category features has received very high attention recently.For one thing,the solution of industry on this problem is artificial feature combinations based on business understanding,with models like Logistic Regression,which spend a lot of manpower.For another thing,nowadays academic research focuses on deep learning models based on feature combinations,which have some flaws such as poor explainability and difficult hyperparameters turning on large-scale dataset.Summarized from the experience on business-based feature combinations in the data mining competitions,an automatic feature combination framework called AutoFeature is proposed in this dissertation.What's more,a heuristic feature combination searching algorithm based on matrix decomposition called MF-AutoFeature is introduced for further improvement of the efficiency in feature construction,and it has been used on tens of millions of data with millions of users.Cascading with GBDT,this framework has excellent predictive capability compared to Factorization Machines(FM),FM's variants and several state-of-the-art deep learning models.Meanwhile,this framework has strong business explainability and there is no need of manual adjustment of parameters during the feature construction.Besides,several latest popular deep learning models have been optimized in this dissertation.The main work is shown below.Based on the DeepFM,the DeepAFM and DeepNFM are proposed in this dissertation,which add the attention mechanism and neural network into the FM part of Deep FM respectively.Experimental result demonstrates the predictive capability of DeepAFM and DeepNFM is better than DeepFM and other deep learning models on tens of millions of data.Our dissertation also integrates sequence information into FM and proposes variants of FM based on Recurrent Neural Network called RFM and DeepRFM,which make sequence information modeling and feature combinations could be done simultaneously.Experiment shows that RFM and DeepRFM have better predictive capability than FM and DeepFM for users which contains rich history records.