Research On Collaborative Filtering Algorithm Based On Matrix Decomposition And Clustering

In recent years,with the continuous innovation of computer technology and the rapid development of network science,many scholars pay more attention to the various aspects of the recommendation system.In the actual network environment,information overload is often encountered.Can play a good role in its relief.Among many recommendation algorithms,collaborative filtering algorithm is one of the most classical and most used algorithms in this field.When the user browses the information on the Internet,a history record or rating is left,and the collaborative filtering algorithm uses the similarity degree to measure the similarity between the user or the item and determine the closest neighbor set of the user or the item according to the information.The final recommendation results need to predict the score of the items that have not been scored in the specified neighbor set data,and then generate corresponding recommendation services for different objects.At present,for a variety of application scenarios and applicable environments,the collaborative filtering algorithm has also been improved and adjusted accordingly,so that the improved algorithm can better improve the performance of all aspects of the recommendation system,resulting in better recommendations;However,with the rapid development of today's Internet,not only does it have massive unpredictable large-scale data,but also has a variety of cumbersome data types.At the same time,it also imposes higher requirements on the speed of processing and calculation of recommended system information.This causes traditional collaborative filtering algorithms to face enormous challenges.The sparseness of big data information and the scalability of the algorithm make the current recommendation system not produce the corresponding recommendation function well.In view of the above analysis,this paper proposes a collaborative filtering recommendation algorithm based on ALS matrix decomposition and improved K-means clustering in the Spark platform based on the research of collaborative filtering algorithms.The sparseness and slowness of the operation caused by the influence of big data background.The specific work is as follows: First,this paper uses ALS matrix decomposition method to perform matrix completion on the high-dimensional,high-sparse matrix composed of massive data,and achieves data preprocessing and matrix filling.This method can support parallel computing better and improve the speed of operation.Secondly,this paper adopts a K-means improved algorithm that determines the center point with the maximum distance,and constructs a clustering model for the filled matrix information.Finally,according to the similarity between users,the nearest neighbor set of the target user in the clustering model is determined,and a corresponding predictive analysis is performed to generate a recommendation;this paper applies the improved algorithm to the Spark platform and uses MovieLens information as data.Sets,parallelization experiments,filling matrices and building clustering models can be performed off-line on the Spark platform,dramatically increasing the speed and reducing the amount of computation on-line;after experiments,the results show that the proposed matrix is based on the matrix.Decomposition and clustering collaborative filtering recommendation algorithm can better alleviate the problems of high-dimensional matrix sparsity caused by big data,making the algorithm have accurate recommendation accuracy,high-speed processing speed and good scalability.