Research On Hybrid Top-N Recommendation Algorithm Based On Big Data

With the proposal of Internet + and the arrival of the era of artificial intelligence,more and more data are generated in the Internet,which is like a treasure to be developed.Properly mined data can bring huge benefits to practitioners.Big data mining technology has increasingly become the research focus of computer science.One of the most popular research directions is the optimization of recommendation algorithm.At present,the recommendation technology of major APPs,such as Douyin and Taobao,ensures the customer loyalty of their APPs.In the field of project recommendation,previous researchers have done a lot of work according to their own experience,such as the proposal of basic algorithm,but the algorithm is always flawed.Four problems in this field,such as data sparsity,cold start,extensibility and user interest drift,have not been effectively solved,and have always been the focus of researchers in this field.Aiming at the shortcomings of traditional recommendation algorithms,this paper proposes corresponding improvements.According to the traditional collaborative filtering recommendation algorithm,a hybrid Top-N recommendation algorithm based on collaborative filtering algorithm is proposed,Top-N analysis method is to get the required N data from the research object through the Top-N algorithm,and select the largest or smallest N data from the sorting list,so as to provide methods for solving the problem.The main content of this paper includes the following aspects:(1)The construction and data set of the big data platform were introduced,and the Hadoop computing platform was downloaded and configured.After the completion of the construction,the effectiveness of the platform was tested.The Movielens data set and IMDB data set required by the experiment were processed.The above two aspects laid a good foundation for the subsequent experiments,The algorithm will be run and tested on the big data platform,which can not only save time,but also reduce the downtime caused by insufficient hardware performance.The data set is mainly used for the learning of the algorithm and the analysis of experimental results(2)Using the principle of cascading algorithm,a kind of mixed attribute clustering is adopted to realize the flexible clustering based on goods,Then,the strong association rule algorithm is used to mine the association relationship between goods,so as to find the goods that users may like,This step addresses scalability issues and user data sparsity issues.Then the weighted method is used to integrate the above algorithm with the time-series collaborative filtering algorithm.The primary consideration of temporal collaborative filtering is temporal correlation,Can solve the user interest drift problem.Finally,The parameter adjustment and algorithm comparison are given.Finally,the hybrid top-n recommendation algorithm based on big data is implemented.The system uses Django in Python to build the page and embed the algorithm in it.