Research On IPTV Users' Behaviors Analysis And Distributed Collaborative Filtering Recommendation Algorithm Based On Spark

Today,the Internet and information technology fills our lives with more and more data,how to provide users accurate and personalized information becomes significant and meaningful to e-commercial.Under such situation,recommendation system is invented.Recommendation system can provide content to users which they may be interested in.This helps users save a lot of time and effort when they have to make choices among great amount of data.Recommendation system has been widely used in online shopping,film and television recommendation and other areas,and it shows a great value in those areas.The input of recommendation system is users' preference information,while in IPTV system,it is difficult to collect users' preferences directly.Thus we have to dig out the preference information through analyzing users' behaviors;this way of gaining users' preferences is called implicit feedback.However,there are few researches on implicit feedback feature selection and data validity analysis.At the same time the video information and users' behavior information in IPTV system grow rapidly.How to deal with the big data of those information has become a challenge to recommendation algorithm.Based on the recommendation algorithm of IPTV,this paper analyzes the data of users' behaviors and designs a data preprocessing strategy,and proposes an implicit scoring model based on users' behaviors.Then,this paper uses clustering algorithm to optimize the user-based collaborative filtering recommendation algorithm.At the same time,combined with Spark distributed platform,a distributed recommendation algorithm is built to adapt big amount of data.The innovation of this paper lists as follows:1)Based on the analysis of users' behaviors,we develop the preprocessing strategy of users' behaviors.Then,an implicit scoring model which could reflect users' preference is constructed based on the users' behaviors.The experimental result shows that the implicit score based on the IPTV users' behaviors reflects the users' preference information well,and the implicit score can be effectively used as the input data in the recommendation algorithm.2)According to the patterns of IPTV users' behaviors,this paper uses cluster algorithm to cluster users.And we combine the cluster algorithm and collaborative filtering recommendation algorithm to improve the quality of recommendation algorithm.Experimental result shows that compared with the common collaborative filtering recommendation algorithm,the quality of collaborative filtering recommendation algorithm based on user clustering is significantly improved.3)Based on the characteristics of Spark distributed computing platform,this paper designs and implements a distributed collaborative filtering recommendation algorithm of IPTV system,which includes preprocessing strategy,implicit scoring model and collaborative filtering recommendation algorithm based on user clustering.Experimental result shows that the time efficiency of algorithm on Spark distributed computing platform is better than that of Hadoop.