Feature Interaction Information For Click-Through Rate Prediction

Non-contiguous and categorical sparse feature data are widely existed in this problem.To build a machine learning system with these data,it is important to properly model the interaction among features.In general,traditional CTR prediction tasks use shallow models because of data sparseness and model optimization issues,such as logistic regression and Factorization Machine(FM).However,those models require large number of feature engineering to represent feature interaction to improve the prediction effect.This requires professionals to spend a lot of time manually constructing features,and the construction process requires strong domain knowledge and the entire process is not universal.This paper aims to fully utilize the information between features and improve the performance of deep neural networks in the task of CTR prediction.At present,the CTR methods model the feature interaction in a relatively simple way,which ignores the magnitude of the influence of each feature association vector on the prediction result of the click estimation,and multi-dimensional feature interaction information is missing.Therefore,in this paper,different models based on the deep neural network are designed for the associated feature weight information and multi-dimensional feature interaction to improve the performance of clickstream estimation.1)In this paper,we propose a factorized weight interaction neural network(INN)with a new network structure called weight-interaction layer to learn patterns from feature interactions and factorized weight parameters of each feature interaction.The proposed INN can greatly reduce the dimension of sparse data via the weight-interaction layer,while the multi-layer neural network can be used to capture high order feature latent patterns;2)It incorporates multi-dimensional features When correlating information,this paper constructs a multi-dimensional feature association relationship information representation based on a 3D tensor decomposition method,where each 2D matrix slice of the 3D tensor represents a relationship.Compared with the traditional CTR estimation method,this paper proposes an end-to-end deep neural network method that incorporates feature association information representation,which can greatly reduce the artificial workload of constructing features and has a good prediction effect.We evaluate our models on CTR prediction tasks compared with classical baselines.Evaluation results demonstrate that feature interaction contains significant information for better CTR prediction.It also indicates that our models can well encode these kinds of feature interaction information into CTR prediction,and achieve better performances in real-world datasets.