Research And Implementation Of News Recommendation System Based On Collaborative Filtering On Hadoop

With the advent of the era of information,the Internet carrying more and more information,which has made it difficult for people to find information of interest when browsing the news portal.The emergence of the content aggregation platform and the recommendation systems is to solve the problem of information overload.The content aggregation platform crawled news from the major news sites,stored on the local system,and then push news to the users on the platform through the recommendation system,for each user to provide personalized news recommending.Conventional recommendation systems are generally based on collaborative filtering algorithms.However,there are partial flaws in collaborative recommendation based in collaborative filtering and user recommendation.The conventional hot recommendation algorithm will attenuate the thermal value of the news with a fixed attenuation factor.In this paper,through the test and analysis found that this algorithm cannot balance the user flow and heat attenuation between the imbalance,will lead to a very low hot spot capture rate.And the conventional user-based recommendation only uses the collaborative filtering algorithm to recommend the user based on the neighborhood user group,which will lead to the inaccurate recommendation when the user's interest changes.Based on the implementation of content aggregation platform,this paper studies and improves these two recommendation algorithms.The main work and innovation of this paper are as follows:1.This article implements a simple but comprehensive content aggregation platform,which includes several subsystems: web system,cache service,crawler,database services and Hadoop cluster,the various subsystems communicate with each other through the corresponding protocol.The content aggregation platform mainly serves the recommended algorithm.2.Aiming at the hot recommendation algorithm,this paper proposes a hot news recommendation of adaptive time decay coefficient,taking into account the single news flow and system traffic,in order to calculate the time attenuation coefficient of each news.Tests show that the adaptive time attenuation coefficient can significantly improve the news hotspot capture rate.Finally,for the new news,we used the potential hotspot mining algorithm to update the hot news.3.Aiming the user-based recommendation algorithm,this paper proposes a userpersonalized recommendation with a auto-improved model.First,this paper analyzes each user's neighboring users,and then generate a separate recommendation list for each user based on the project-based recommendation.Then,this paper analyzes the user's browsing history,using the correction algorithm to regularly modify the user model to track the changes of the user's interest,to provide better recommendations.Test results show that the modified algorithm can complete the modification of most of the user model in three iterations.The improved user-based recommendation algorithm can provide better precision and recall rate than the conventional algorithm.Finally,this paper combines with the hot news recommendation,achieves the user personalized recommendation to tap the user's potential interests.