Research On Collaborative Filtering Recommendation System Based On Spark

In the era of big data,it is very important to extract information of users' interest from massive data with recommendation system.However,the recommendation system in single mode is inefficient when dealing with big data.With the increase of data volume index,the time cost of the recommended process is unacceptable to users.To solve this problem,this paper studies the collaborative filtering recommendation system based on Spark.There is an improved Alternating Least Squares(ALS)recommendation algorithm based on popular weight and a collaborative filtering recommendation algorithm based on Recurrent Neural Networks(RNN)being proposed.They are applied and verified on the Spark platform.The research mainly includes the following aspects:Firstly,with regard to sparseness of the scoring matrix data,the recommendation method based on ALS is studied.However,the current proposed ALS ignores the preference information implied by the missing score under the popular item,and does not explore the influence of the implicit feedback in the popular item on its recommended weight.Therefore,this paper proposes to apply popular weight to ALS algorithm.The prevalence evaluation formula determines the weights of unscored missing items,and combines the weights into the objective function of ALS to suppress overpopulation of popular items.Secondly,the current recommendation system is mainly a static recommendation,which means that the user's interest is fixed and does not consider time's influence on the user's interest.Therefore,this paper mainly studies dynamic recommendation methods based on two kinds of RNN,namely,the dynamic recommendation method based on Long Short Term Memory(LSTM)and the dynamic recommendation method based on Gated Recurrent Unit(GRU).The proposed dynamic recommendation method uses RNN's deep learning and dynamic prediction capabilities to achieve dynamic recommendation.Regarding that RNN predictions of users' future long-term interest are not ideal,this paper integrates the idea of matrix decomposition into the RNN network,because the matrix decomposition considers the global characteristics between users and projects,and performs well in the static long-term recommendation system.Therefore,they are combined to improve the long-term prediction ability of RNN.Thirdly,the ALS recommendation system based on popular weight is applied on the Spark parallel processing platform,so does the dynamic recommendation system based on LSTM,the dynamic recommendation system based on GRU,and the collaborative filtering dynamic recommendation system based on matrix decomposition and RNN.Fourthly,in order to verify the recommendation effects of proposed algorithms,they are compared with neighbor-based recommendation system and ALS-based recommendation system.The experimental results show that the improved ALS algorithm is more accurate than the traditional ALS algorithm in predicting scores,and both the accuracy and the recall rate are improved;the improved RNN algorithm enhances the ability of long-term prediction,and the proposed dynamic recommendation has greatly improved the coverage.