Design And Implementation Of A News Recommendation System Based On Spark

With the development of the Internet,users generate a large amount of data on major websites every day,causing data overload.It becomes very difficult for users to find their own information in massive amounts of data.Search engines and classifieds can help people afterwards.It is more convenient to obtain the desired information.However,these two solutions cannot effectively solve the user's needs,and require the user to actively search for or search in the corresponding classification system.The recommendation system analyzes the user behavior to analyze the user's personal interests,and then the system Personalized information can be recommended to users based on their personal interests.The news recommendation system is an application of the personalized recommendation system in the news field and can be personalized based on the user's different interests.The system first elaborates the theory of the recommended technology,then introduces the related technologies,and performs system design and detailed design from several aspects such as cold start,content portrait,user portrait,hot news,hybrid recommendation technology,and recommendation engine.The cold start adopts Weibo cold start and heuristic logic.The content portrait is to extract the feature vector of the news article.The user portrait extracts the user interest tag according to the user click browse news,the hot news is the real-time statistics daily news traffic,and takes out topK news as The hot news is recommended,and the mixed recommendation technology adopts a combination of a collaborative filtering algorithm that incorporates changes in interest and a content recommendation algorithm that incorporates changes in interest and hotspots and reduces weights.The recommendation engine is a recommendation list that obtains mixed recommendation technical results,hot news,and the like.Recommend to users.In addition,the Spark distributed computing framework is used for high-volume offline training recommended by the news,and the Spark Streaming real-time computing frameworkis used to recommend personalized news in real time.The related data storage system uses the HBase distributed storage system for storage,and uses redis as a cache to enhance the system.operating efficiency.This article's news recommendation system can meet the user's personalized needs,according to the user's own behavior records,extract the user's interest for the user to recommend their own news of interest.