A Neural Network Framework Over Multi-field Categorical Data

Personalized service plays an important role in Internet applications,which forms on the basis of user response prediction.These years,user response prediction has becoming more and more popular in industrial applications.The feature values of user response prediction usually appear in a multi-field categorical format,and exposes a sparsity issue,which brings about a big challenge in learning and prediction.Through detailed and in-depth analysis,we find a coupled gradient issue of latent vector based models,and an insensitive gradient issue of neural network models.To solve the coupled gradient issue,this work proposes kernel product operation to extend vector inner product operation to learn feature interactions,which handles the coupled gradient issue theoretically.To solve the insensitive gradient issue,this work proposes to pass both feature representations and feature interactions to the following classifier and conduct prediction.This work proposes a novel neural network architecture,with high flexibility and expressive ability.The proposed model has been evaluated on several public CTR benchmarks,and shows outstanding performance.Besides,this work has also implemented a deep recommender system,which has been deployed in Huawei App Market.This system achieves 35% CTR improvement on average in online A/B tests.And this work has been published in CCF-A journal,ACM Transaction on Information System.