Research On Personalized Recommender Algorithm Based On Statistics

With the rapid development of the Internet, information has exploded in growth. Data mining has became a vibrant researching area as the big data develops. Powerful driving of commercial interests will force it to continue to promote its development. Personalized recommendation is one of important technology which data mining is used on the Internet. Faced with massive data, recommendation system can achieve a win-win for information consumers and producers. Collaborative filtering algorithm is one of the most successful and the most widely used algorithms among personalize recommendations. However, it depends on the history data of users, so there are some problems such as cold start, data sparsity and so on.Under the new situation of big data, all data mining algorithms, including personalized recommendations, bring the opportunities and challenges to Statistics. On the one hand, data mining algorithms have many ideas from the Statistics, on the other hand, data mining shows a strong vitality. Accordingly, the article is wrote to explore the effect when statistical analysis applied to personalized recommendation algorithm in the case of large amount of data and other data mining models such as the association rules, clustering and other methods to improve the model.This paper presents a personalized recommendation based on statistics, mainly using MATLAB, SAS to do auxiliary programming, and bring out descriptive statistics, multidimensional association rules and collaborative filtering algorithm to recommend.To improve existing shortcomings of collaborative filtering model, this paper present to use one-dimensional and two-dimensional statistics to improve the data sparsity problem, then use SQL SERVER2005and EXCEL data mining module to set up a clustering model. Finally, singular value decomposition method is used to improve the algorithm, and use average absolute error to measure the effect of various improvements. By comparing the conclusions we find that using the results of statistical analysis based on user ratings and characteristics to improve collaborative filtering have better effects. Combining statistics, data mining models for cold-start problems are greatly improved. Experiments described in this paper can be thought that statistical theory can be showed in a variety of complex models. In the way of the future development of big data, statistics not only maintain the most basic vitality, but also strengthen its application in other disciplines, promote statistical methods reform, expanse the depth and breadth of statistical studies of specific scientifical studies.