Design And Implemetation Of Collaborative Filtering Algorithm For E-Commerce

With the advent of the era of big data,data development has an explosive growth rate,how to excavate the value of the hidden depth from huge amounts of data mining is a study of far-reaching significance.In the 13 th Five-year plan,the country will put big data analysis and application in the national strategic level,in other words,the government think the data is value.Under the background of big data,the development of e-commerce has entered a more diversified era,website backstage users can analyze consumer behavior or consumer browsing history,set user preferences model.Combined with the recommendation technology,the website backstage will recommend good products to customers,thereby reducing consumers search cost for right goods.Eventually it will relieve the information overload problem.In e-commerce sites,vast amount of data will be produced everyday,so the recommendation system need excavate and analysis vast amount of data.How to response to user demand fast and accurately requests the recommendation system has strong ability of data mining analysis.Combined with the classic algorithms of data mining,this paper deeply studies the several recommendation algorithms,and has made the improvement in view of the insufficiency of these algorithms.The main research work includes the following aspects:(1)This paper studies the concept? classification and operation process of data mining technology,analyzes the summary?type and data source of Web data mining based on basic data mining,deeply studies the basic concepts?advantages and the principle of collaborative filtering technology operation process.(2)This paper studies the collaborative filtering algorithm based on User(User—based CF)and collaborative filtering algorithm based on Item(Item—based CF)in detail.The two algorithms in recommendation systems were used in the earliest and were the successful recommendation algorithm.In e-commerce sites,User—based CF algorithm is mainly used for analyzing the users relationships and also for excavating,analyzing the similarity among users to make recommendation;Item—based CF algorithm can be used to analyze the user's historical records of consumption.This paper is to realize the combination of these two kinds of recommendation algorithm.(3)Based on limitations for single recommendation algorithm,this paper puts forward the improved method of matrix decomposition and three kinds of hybrid recommendation technologies,including hybrid recommendation framework of the period of combination,the weighted type hybrid recommendation technology and the waterfall hybrid recommendation technology.By optimizing the several hybrid recommendation technology,the system can effectively solve the problem of cold start and sparse.(4)This paper studies the Spark distributed computing framework for big data processing,uses the Spark's programming model and achieves combined collaborative filtering algorithm.Combined with the advantage of Spark parallel computing,the efficiency of algorithm of recommendation is improved obviously.(5)Using the number of indexes to evaluate the recommendation algorithms.The data set is divided into training set and test set,design the recommendation system evaluation experiment and analyze the experiment results in detail.