Design And Implementation Of Personalized Video Recommendation System Based On Hadoop

With the development of computer and Internet technology, data on the Internet expand rapidly and video data also increase at exponential scale. Video data are rich in content, huge in quantity and various in structure. And all these characteristics bring great challenges to the user retrieval in a short time, which makes an overpowering demand for personalized service. Therefore, personalized recommendation arises at the historic moment. How to rapidly and accurately recommend the videos users really interested in from the massive video data and achieve the goal of improving user retrieval efficiency become a problem to be solved.The main work in this paper are:Firstly, establish video data preprocessing algorithm. The algorithm is based on the original video parsing, cleaning, discreting, normalized processing, providing a neat and normative source data for the video vectors’ construction. Do text analysis of news video titles by using Chinese word segmentation and word frequency statistics. Make association rules and affiliate the video data provided by the system and the ontology data by web crawling, which enrich the data dimensions and build video feature vectors for the recommendation.Secondly, through data filtering, filter the redundant data which are irrelevant to the watching behavior. Through the behavior analysis and modeling processing, dig out the user’s interests in the category, country, star, director, heat value and the score dimensions or key words dimension, to establish the double tree user interest model based on video features for the recommendation.Thirdly, establish a multidimensional recommendation algorithm based on the content, and the algorithm calculates the user interest model and video feature vector similarity of each dimension by cosine similarity formula. Along with the weight values set for each dimension, achieve the user’s preference value of each video, completing the TOP-N recommendations. Add a mixed strategy to improve the traditional collaborative filtering recommendation algorithm based on the project. The improved algorithm adds the latest and popular video list to the primary recommendation list, solving the real-time and hot spot ignorance problems exist in the news video recommendation.Forthly, accomplish the personalized video recommendation system based on the Hadoop distributed platform. After testing the system has reached the design requirements and successfully applied in a smart TV.