Design And Implementation Of Personalized News Recommendation System Based On Hadoop Platform

This is an era of information pursuit.With the popularization of Internet technology,people's access to information is becoming easier and easier.In order to satisfy people's thirst for information,a variety of news and information applications emerge in endlessly.Network news can accommodate a large amount of information through news display,hyperlinks and other ways.News is also more abundant and diversified.However,the rapid growth of information data on the Internet at an explosive speed has produced a lot of spam information and led to the emergence of information overload and becoming more and more serious.Information overload has slowly become a huge obstacle for people to quickly and efficiently obtain useful information,which leads consumers to want to find information they are interested in from a large number of information(items),and information producers think it's hard to make your production information stand out and get attention.The task of recommendation system is to connect users and information(items).Under the impetus of information overload,recommendation system has become a necessary tool for major Internet companies to explore new territory.Personalized news recommendation system is to solve the problem of information overload.In order to achieve accurate personalized service,the first step is to model users and improve user portraits.Secondly,we use existing recommendation algorithms as recall operations and add business-compliant recall methods according to actual business needs to provide better recommendation underlying data for personalized recommendation.Then we use logistic regression model of machine learning algorithm to predict click-through rate.We used optimization algorithm and feature extraction methods to optimize the model,which improved the accuracy and generalization ability of our model.Finally,we combined user portraits and recall data and used the model to predict.In order to reduce the deviation of prediction results,we used sequential regression to correct the prediction results.Less error.In order to make users have a better experience,we deploy the whole system on Hadoop platform and use Spark for data operation,which greatly improves the response speed and stability of the system.At present,some functions of the system have been put on line.The speed response of each recommendation is less than l second,and 10 personalized news articles are recommended to users.The click-through rate of news is 15%to 20%.The accuracy of the model is relatively high,which basically meets the system positioning requirements.