| With rapid development of information techonology,recommendation system plays more important role in human's daily life.Especially as the information grows explosively,people are getting hard to search the required information from the massive data.Therefore,it is urgent to research the intelligent and personalized recommendation system for large-scale scenes.After decades of researches,there exist a variety of of recommendation algorithms and systems,which are suitable for different types of data.People can choose appropriate algorithms for different types of data for recommendation.On the one hand,the existing recommendation algorithms and systems often only have local implementation and are just appropriate for the small scale data.They are difficult to be applied to the scene of massive data.Therefore,in this paper,we design and implement parallelized algorithms on Spark for the traditional shallow-layer recommendation system.We have implemented the parallelized nearest-neighbor-based recommendation algorithm which is suitable for dense and low-rank data.For the sparse rating data,we have implemented the parallelized SVD-based recommendation algorithm.For multi-view data,we have implemented the parallelized MVM recommendation algorithm.These algorithms are multi-view and shallow-layer.The nearest-neighbor-based algorithm uses user rating vectors or item rating vectors as the feature vectors,which is a single-view and 0-layer algorithm.The SVD-based algorithm uses the decomposition mechanism to extract the features of both users and items,which is a 2-view and 1-layer algorithm.The MVM algorithm uses decomposition mechanism to extract the features for every view from the multi-view data,which is a multi-view and 1-layer algorithm.On the other hand,as the intelligent equipments develop,the type of information is diversified.The traditional recommendation algorithms are difficult to adapt the complex real-world data,such as text,pictures,vedios and other information.In this paper,we study and design a series deep recommendation models on TensorFlow for the real-world data.We use the deep nerual network to extract the collaborative filtering features of both users and items.We also design a deep content filtering model to utilize the information in complex data such as text and picture.To combine all the available information from the real-world data,we design a deep multi-view model which makes full use of all the information.Finally,we also design a parallelized scheme which is suitable for all the deep recommendation algorithms in this paper.The deep recommendation algorithms in this part use the deep nerual network to extract the features of the respective views,which belong to multi-view and deep-layer algorithms.For multi-view and shallow-layer recommendation algorithms,the main purpose of this paper is to improve the efficiency of the corresponding algorithms in term of the accuracy.For the nearest-neighbor-based algorithm,the parallelized algorithm obtains 15x speedup compared to the single-machine method.For the SVD-based algorithm,our parallelized algorithm reaches 30x speedup.For MVM,the parallelized algorithm obtains 40%performance improvement compared to the parallelized Graphx version.For multi-view and deep-layer recommendation algorithms,our purpose is to improve the accuracy of the recommendation algorithms and also make them available for big data scene.For deep collaborative filtering model,proposed algorithm obtains 0.43%-1.93%improvement on accuracy compared to other models.For deep content filtering model,proposed algorithm obtains 0.14%-2.3%improvement on accuracy compared to other models.Compared to deep collaborative filtering model and content filtering model,proposed deep multi-view model obtains 0.49%-1.89%improvement on accuracy.Experiments are also conducted to evaluate the performance of our parallelized algorithms for deep recommendation and experimental results show that proposed parallelized algorithms work with good parallelization performance. |
PDF Full Text Request