Design And Implementation Of Video Real-time Recommendation System Based On Multi-algorithms Fusion

21st century social entertainment industry has been rapidly developed,and video as a vivid image of the information carrier is favored by the people.The rapid expansion of the video business brought about by the expansion of the explosive video data allows users to browse the video that seems confused,and with the rapid growth of users,finding the users in a short time to match the video is not evident.The traditional recommendation system has some shortcomings:First,the traditional recommendation system is mostly developed on the stand-alone recommendation system and the MapReduce calculation based on the Hadoop architecture.When the data volume is big,the former can not achieve greater throughput,the latter is based on the persistent storage of the computing model can not handle the iterative requirements of the recommended algorithm and to ensure effective computing efficiency.Second,the traditional recommendation system can give the user a personalized recommendation,but does not update the recommendation list without collecting the user action information in real time when the user's interest may fluctuate at any time.Furthermore,the traditional system does not fully consider the short boards of the various recommended algorithms in implementing a list of recommendations that typically generate only a single algorithm.In order to solve the above problems,this thesis focuses on the real-time personalized recommendation of real-time user feedback and real-time recommendation results update,learn from NetFlix recommended system of three-tier architecture to build a Spark platform based on the general off-line,near line and online Recommended system.Based on the above system,the similarity recommendation model based on content feature,the similarity recommendation model of user characteristics and the recommendation model based on display feedback are proposed,and the corresponding incremental model is put forward to provide real-time recommendation function.And in order to take into account the user's long-term preferences and improve the accuracy of real-time recommendation algorithm,this thesis presents a real-time recommendation algorithm based on user preferences.Finally,this thesis presents the convergence strategy of the recommended results of various recommended algorithms to make up the short board of each recommendation algorithm.The main contents of this thesis are:Firstly,this thesis fully studies the computational principle of Spark and the processing mechanism of Spark Streaming.It analyzes the scenarios and business processes of the data collection framework Flume and Kafka,and combines the support of massive data Distributed storage,random query faster HBase.On this basis,a general recommendation model based on Spark platform is constructed,which makes the recommended algorithm of incremental support can run independently on the model.Secondly,this thesis studies the traditional content recommendation and collaborative recommendation algorithm implementation process,uses IF-DIF technology to describe the video description information and the user's label data of the video as the video feature vector and calculate the similarity of the content based on the content Recommended recommended list.In this thesis,the user matrix and the content of the content matrix are obtained by decomposing the ALS matrix of the user's score matrix data and the cooperative recommendation list is given.In addition,the real-time updating model based on the content recommendation and the user collaborative recommendation is given.Thirdly,this thesis proposes a recommendation algorithm based on user preference,which integrates the long and short term preferences of long and short-term historical preference real-time preferences for the collected user feedback data batch.In addition,this thesis carefully investigates the advantages and disadvantages of the above algorithm,and combines various algorithms to overcome the above metioned shortcomings.Finally,through the deployment of Hadoop,Spark and other distributed software,a platform has been implemented.The experimental results show that this system can meet the requirements of real-time recommendations and effectively improve the accuracy of the recommended system.