Research On Collaborative Filtering Algorithm Base On Multinomial Finite Mixture Model

With the rapid development of the Internet, the information overload of the Internet has become an important discussion topic. The key issue of studying this topic is how to filter out the valuable information from the mass information. To solve this problem, the personalized recommendation shows its academic significance and practical value as a kind of key technology. Collaborative filtering and collaborative filtering derivatives algorithms have been adopted by a large number of Internet companies and become a hot spot of academic research for the characteristics, which are independent of the specific content of the information and easy to implement, and which can produce novel recommendations. However, there are still some problems in collaborative filtering such as low precision and poor scalability. In this paper, the finite mixture model and the traditional collaborative filtering algorithms were combined to explore a new personalized recommendation technology. The following research contents are included.Firstly, the concept of personalized recommendation was introduced. The appearance and current situation of the development of personalized recommendation was summarized. An overview of the concept, statistical principles, and characteristics of finite mixture model has been offered. The theoretical basis of this paper, solving algorithms of finite mixture model and applications in personalized recommendation of finite mixture model has been study.Secondly, several advantages and disadvantages of collaborative filtering algorithms were summarized. The multinomial finite mixture model was used to model the data set and the cluster result was used to optimize the Slope One collaborative filtering algorithms. The multinomial finite mixture model cluster algorithms improved the precision of the recommendation, and the MML(minimal message length criterion, MML) enhanced the efficiency. Experiments, in which the standard recommendation data from MovieLens were used, confirmed the improvement of the new algorithm with a better MAE (mean absolute error, MAE)Finally, in order to meet the personalized recommendation of big data trends, MapReduce implementation method of Slope One algorithm based on multinomial finite mixture model was proposed. Iterative MapReduce process was used to achieve the finite mixture model parameter estimation and decomposition Slope One algorithm into two MapReduce process, and the whole distributed algorithm was completed ultimately. The reliable precision and excellent scalability of the algorithms on Hadoop platform was verifed by the experiments.