Personalized Micro-blog Recommendation Based On Collaborative Filtering Technology And Its Implementation In Spark Framework

With the rapid development of Internet technology,the amount of information on the Internet has increased exponentially.Entering the era of big data,it has directly triggered the information revolution of big data.How to extract and mine data in such a large amount of big data environment and get the information of interest is an urgent problem to be solved in the era of big data.This thesis is focusing on both theoretical research and practical application.Recommendation algorithm is an important way to solve item recommendation in large data environment.Traditional recommendation algorithms are based on user-item characteristics for collaborative filtering calculation,lack of consideration of environmental factors,time factors and other factors,resulting in low recommendation accuracy.In this thesis,a new improved personalized social recommendation algorithm is proposed,which is Collaborative Filtering with Time Social Exposure(TSERec)based on time characteristics,which integrates social exposure and social timeliness into the collaborative filtering algorithm.Different from the traditional collaborative filtering algorithm,social information is integrated into the user-micro-blog matrix of the evaluation model.This algorithm uses social information and social timeliness to calculate the exposure degree of social connection.On the basis of improving the personalized social recommendation algorithm,we have engineering practice for micro-blog large data application scene,mainly using Spark Streaming technology and Spark framework as the core technology to implement the algorithm,so as to realize online recommendation based on flow computing.In the process of engineering practice,it is built in accordance with the idea of software engineering,including the components of architecture design,system design,database design,system implementation and system testing.The results of the system test fully confirm that the improved recommendation algorithm can adapt to the big data application scenario and has better recommendation effect.