The Design And Implement Of Combined Recommendation Algorithm Based On Micro-blog Information

In the era of information explosion, recommendation algorithm is very important to overcome information overload and help knowledge discovery. With the data dimension becoming more and more complex, single recommendation algorithm gradually cannot meet the demand for personalized recommendation problem day by day. This paper starts from micro-blog user recommendation problem, has deeply studied recommendation algorithm and the related knowledge, and has done a lot of optimization to the cold start problem in recommendation algorithm, has improved the recommendation accuracy compared to the single recommendation algorithm. The main work has been done as follows:1.Model design:first, this paper builds the user clustering model based on demographic characteristics, which is a powerful tool for cold start user recommendation; then, facing the problem of micro-blog data could not be scored, feature engineering is built reasonable, the content popularity model based on user group is put forward and the user rating matrix is established; then, this paper uses the latent factor model to predict score efficient; finally, the paper proposes the user group Top-N recommendation model based on KNN, and then combines the model proposed above together to get the combined recommendation model, the result of the experiment shows that the model proposed in this paper is much better than traditional single recommendation model.2. Algorithm design and implement:considering from efficiency, this paper parallels optimization the KNN algorithm and k-means algorithm, and implement them on the Spark platform. Considering that the KNN algorithm needs a huge search space, runs slowly, the KNN algorithm based on Broadcast KD tree is designed, which improves the efficiency of the algorithm greatly.3.Algorithm evaluation and optimization:in the algorithm evaluation and analysis, this paper uses the R language, which could visualize analysis the various stages of data efficient, and on this basis, a common recommendation evaluation method is improved, the evaluation model based on analytic hierarchy process is proposed, so the evaluation indicators becomes more fit to the needs of practical application; in the algorithm optimization, the paper uses the elbow method to obtain learning curve, which can better detection the prediction ability of the algorithm.4.Platform build and optimization:facing the problem of huge scale data, in order to analysis and calculate effective, this paper sets up the Spark and Hadoop clusters, and optimization the memory and parallelism the Spark cluster for a certain degree, which enhances the study efficiency.According to the micro-blog user recommendation problem, this paper tries to build the big data platform, design the model, implement the parallelism and optimize evaluation, compared with the traditional platform single model, the accuracy, recall, coverage and F1score of the model designed and implemented in this paper have been improved significantly, which has an important significance to the analysis and research to other recommendation problem.