Research On Hybrid Recommendation Algorithm Based On Improved Collaborative Filtering And GBDT

With the rapid development and popularization of the Internet,more and more people will use the Internet to obtain valuable information,such as online shopping.Users can search for and obtain their favorite products through the data of e-commerce platforms.However,with the development of e-commerce and the increase of users,the massive amount of information generated will cause the problem of information overload,which will cause serious interference to users' choices.Commodity recommendation algorithms can help users quickly obtain valuable information and effectively solve the problem of information overload,which is a current hot research direction.The recommendation algorithm can not only filter through massive data and actively recommend information of interest,but also make personalized recommendations for users,simplifying the user's acquisition process of target products and saving time.In order to improve the accuracy and efficiency of the recommendation system,this thesis designs a hybrid recommendation algorithm based on improved collaborative filtering and GBDT,this algorithm idea of the algorithm is to predict user purchase behavior based on user behavior data,and a new algorithm model for predicting user purchase behavior is designed.Research content includes: 1)Data preprocessing is used to improve the quality of the original data,and to avoid the problem of large differences between positive and negative samples through sample selection.2)Feature engineering mainly analyzes from three aspects: feature extraction,feature selection,and construction of feature systems.3)Recommendation algorithm model design,this article will select collaborative filtering and decision tree hybrid algorithm to make the recommendation model.For brand data that interact with users,a random forest derived from the decision tree is used to process it to avoid overfitting of the decision tree.For the data of brands that have no interaction with users,an improved collaborative filtering algorithm is used for processing,and the user's behavior is given the weight of hierarchical analysis in the calculation process to improve the rationality of the collaborative filtering algorithm.Then the recommendation results of random forest and collaborative filtering are mixed through the combination of hybrid algorithms,and the mixed recommendation results are input into the GBDT model for further optimization,making the recommendation algorithm model designed in this paper more perfect.In order to verify the effectiveness and superiority of the hybrid recommendation algorithm,this paper uses the current mainstream large-scale data processing computing engine Spark development environment for implementation and group verification.The thesis introduces the design and implementation architecture of Spark recommendation system in detail,which includes monitoring module,data storage module,offline calculation module,recommendation engine module and so on.At the same time,the database design of the system implementation and the construction process of the feature system are given,Finally,it focuses on the comparison experiment between the hybrid recommendation algorithm and the traditional single recommendation algorithm and the comparison experiment of the hybrid recommendation algorithm operation response based on different platform environments.The experimental results data show that the hybrid recommendation algorithm designed and implemented in the Spark environment has good effectiveness and superiority in recall rate,accuracy rate,responsiveness,etc.,and improves the user's shopping experience,At the same time,the hybrid algorithm provides a good reference model for the design and implementation of other similar recommendation algorithms.