Research On Hybrid Movie Recommendation Algorithm Based On Bias LFM And LSH

With the popularity of the internet,various data online are growing exponentially.While these data bring convenience to people,they also bring the problem of "information overload".At present,personalized recommendation technology represented by content-based and collaborative filtering has become an important means to solve this problem,which enables people to efficiently obtain the information they need from mass information.However,with the advent of the era of big data,the development of collaborative filtering recommendation algorithm also faces with many challenges,mainly for two reasons:1.Data sparsity problem.2.Scalability problem.These problems are the key problems in collaborative filtering recommendation systems.In order to solve these problems,this thesis proposes a hybrid method of time bias latent factor model and local sensitive hash to improve the accuracy and efficiency of movie recommendation.The main works of this thesis are as follows:1.In order to improve the prediction accuracy problem caused by data sparseness,based on the reality of movie recommendation,this thesis adds the time bias of users and movies on the basis of the Latent Factor Model(LFM)algorithm,and modifies the loss function of LFM.This thesis considers the influence of users on movie scoring habits over time and the influence of movie popularity on rating.It mainly simulates the user's rating habits based on the Ebbinghaus memory curve,and modifies the movie rating based on the relationship between movie rating and movie popularity to improve the loss function and improve the accuracy of rating prediction.2.In order to improve the scalability problem,this thesis uses the Local Sensitive Hash-ing(LSH)algorithm to reduce the dimension of the matrix to reduce the search range of the data.In the recommendation system,the LSH algorithm can map similar users into the same hash bucket as candidate users with a higher collision probability.This method can filter out a large number of dissimilar users to avoid unnecessary similarity calculation,so as to quickly obtain similar users in neighbors.In this thesis,an index structure of similar users is constructed through a LSH algorithm.The target user's score vector is used as input,and the output is a collection of similar users.The time consumption of finding the set of similar users is approximately constant time,which will improve the recommendation efficiency.3.The experiments done in this thesis are all verified using the MovieLens data set.By comparing the traditional Singular Value Decomposition(SVD)and Funk SVD algorithms,and analyzing the experimental results of each Mean Absolute Error(MAE)and Root Mean Squard Error(RMSE)respectively,it can be concluded that taking the time factor into account,the LFM has a better prediction accuracy than the other two matrix decomposition algorithms.And it can be seen that when the latent factor value is greater than 60,the use of the LFM can improve the accuracy of the recommendation.Through the control of the number of hash functions and hash tables in the LSH algorithm,it can be seen that the increase of the number of hash functions will cause the stricter calculation of similar users in neighbors and thus lower the accuracy.And with the increase of the number of hash tables,the conditions for searching for similar users in nearby neighbors becomes more relaxed,and more similar users will have a great probability to fall into the same hash bucket.By comparing whether the LSH algorithm is added to the user-based collaborative filtering,it is observed that when the LSH algorithm is added,the time consumption becomes much lower compared with the traditional user-based collaborative filtering algorithm and the accuracy is not much different.The experimental results prove that for the traditional matrix factorization algorithm,adding the bias of users and items to the loss function can better improve the effect of scoring prediction.At the same time,the method of constructing an index structure for the user-movie matrix reduces the time consumed to find similar users,thereby improving the scal ability of the recommendation system.