Research On The Application Of Learning To Rank In Recommendation Systems

Learning To Rank(LTR)is a supervised machine learning method used to predict the order of elements in the list.In traditional recommendation system,after recalling recommendation list by collaborative filtering or user model,recommendation list are sorted simply using the matcheddegree between user and recommended item.According to the user click log,LTR train the sorting model which is more scalable and can take various factors into account,to sort the recommended list.This dissertation aims to explore the application of Pointwise Learning To Rank method in the news recommendation system,focusing on how to improve the efficiency of sample processing and feature processing under large-scale dataset,and how to improve the recommendation effectiveness of the Pointwise model.The study includes the following four aspects:(1)For large-scale data in the news recommendation system,this dissertation uses the MapReduce framework of Hadoop to join the user log and news data into a sample that can be trained by the Pointwise Learning To Rank model,and use the secondary sorting mechanism between Mapper and Reducer to improve the efficiency of data join.(2)After features are extracted,the Minimum Description Length Principle(MDLP)is used to discretize the continuous features.The experimental results show that the continuous feature has a higher Single-Feature-AUC value after discretization.(3)For the problem that Filter method of feature selection is fast but the effect is poor,and Wrapper method of feature selection is effective but the computational complexity is high,this dissertation proposed a one-way circulation method of feature selection combined with Filter and Wrapper method.This method consists of two stages.The first stage is the Filter process,to calculate the Single-Feature-AUC value of each feature,and filter the feature whose SingleFeature-AUC value AUC is smaller than specific threshold.The second stage is the Wrapper process,to sort the feature set obtained from the first stage by the Single-Feature-AUC value,and using one-way circulation mode repeatedly to evaluate feature along the order until the feature sub set is no longer updated.The experimental results show that this method obtains the same optimal feature subset with the backward search of greedy strategy,and the AUC value is 2.9% higher than that of the original feature set,and the method greatly reduces the computational complexity is only 33.3% of the backward search of greedy strategy(4)After feature selection,this dissertation combines user feature and news feature to enhance the personality.The experimental results show that the AUC of the model increased by 3.6% after adding this kind of combination feature.