Research And Application Of Classification Prediction Algorithms Based On Deep Learning

In information retrieval,there are a large number of multi-field categorical data in the click-through rate prediction and personalization recommendation applications.For this type of data,there are multiple fields,and each field data has no explicit dependency on other fields.Unlike continuous data of images and speech,such data typically has high dimensional sparsity after processing,and there is a combinatorial relationship between features of different fields.How to extract this complex combinatorial features is critical to improve the performance of such systems as ad click rate prediction and recommendation.Traditional machine learning methods rely on tedious and complex manually crafted combinatorial features to deal with such problems.Deep learning,with its powerful representation learning ability,is good at learning complex relationships in high-dimensional data,and can better extract high-quality features in an end-to-end mode.In this thesis,the deep learning model based on click-through rate prediction algorithms are studied and improved.A network structure of factorization machine and residual network in parallel based on attention mechanism is designed and implemented,and tested on the public dataset.The thesis makes contributions as follows:(1)Wide&Deep model and its variants are researched.The main idea of these models is to extract the combinatorial features of low-order and high-order by combining linear model and deep model.Through experimental and theoretical analysis,the optimization method of the above models is obtained.(2)An enhanced FM&ResNet model is studied to further improve the ability of wide&deep model to extract complex combinatorial features FM&ResNet model,a parallel model structure of factorization machines and the residual network,is designed and implemented.The model introduces the attention mechanism in the factorization machines to assign weights to the different combinatorial features,and introduces the self-attention mechanism to model combinatorial features in the residual network part.By introducing the structure of residual connection,the convergence of the model is better.(3)The model is tesed on public data sets.The experimental results show that the AUC(Area Under Curve)performance of the model is improved by 0.1%on Criteo dataset and 2%on Frappe dataset without significantly increasing training time.The introduction of attention mechanism can effectively improve the performance of the model.In addition,the model can also achieve good results in click-through rate prediction on kuaishou company's short video dataset.