The Improvement And Implementation Of Latent Factor Model Recommendation Algorithm In Big Data Environment

With the rapid development of information technology,the information resources on the Internet have become more and more abundant.However it comes the problem of information overload at the same time.At present,the information filtering technology is the main technology to solve the problem of information overload,and the recommendation algorithm is a typical information filtering technology,which has a wide application prospect in Internet information service.However,with the advent of the era of big data,the adaptability of various recommendation algorithms has received wide attention,and many recommendation algorithms have shown limitations and incompatibility with big data application environments.Therefore,it is becoming increasingly important to study the recommendation algorithm adapting to big data environments.At present,among the variety of recommendation algorithms,the Latent Factor Model recommendation algorithm has become a very important recommendation algorithm because of its good adaptability to big data environment.Therefore,this thesis carried out a systematic analysis and research,focusing on Functional Gradient Boosting Latent Factor Model(FGBLFM)recommendation algorithm and its distributed realization.The main research work and achievements of this thesis are as follows:1?This thesis makes a systematic analysis and research on the current mainstream recommendation algorithms,including Content-Based recommendation algorithm,Collaborative Filtering recommendation algorithm,hybrid recommendation algorithm,SVD Matrix Decomposition recommendation algorithm and Latent Factor Model recommendation algorithm.On the basis of carefully analyzing and studying the principle of the above-mentioned recommendation algorithm,the Latent Factor Model recommendation algorithm is selected as the key research object of this thesis.2?This thesis designed an improved algorithm(Functional Gradient Boosting Latent Factor Model,FGBLFM)against the problem that Latent Factor Model is difficult to train to get the optimum solution due to too many parameters in the big data environment.The new algorithm can get the better parameters than the traditional algorithm by using the functional gradient Boosting strategy and improve the accuracy of the conjecture score.3?The thesis makes a deep analysis and research on the current mainstream distributed processing platform Spark,and implement the improved Functional Gradient Boosting Latent Factor Model on Spark,which verifying the feasibility of the algorithm.4?The thesis designed the experiment to verify the performance of FGBLFM,using MovieLens as the data set,RMSE and MAE as the evaluation index,User-Base Collaborative Filtering,Item-Base Collaborative Filtering and Latent Factor Model as the contrast object.The results of this thesis have a good reference value for the further study of the recommendation algorithm in the big data environment.