The Sparsity Research In The Recommendation Engine

Recommendation engine provides users with personalized recommendationsthrough sending interest information and the application of machine learningknowledge. Collaborative filtering is one of the most common personalizedrecommendation technologies in recommendation system fields, it is applied widelyand has become an very important algorithm in the field of recommended. Existingcollaborative filtering algorithm, however, has a very common and significant problem:Data sparsity. Since it is common that the users’ evaluation of project is less than1%inthe application recommendation engine scene, information matrix established betweenthe user and the project is sparse. And information matrix is the base of collaborativefiltering techniques to recommend data, how to solve data sparsity problem isparticularly important.This paper is intended to discuss the data sparsity for collaborative filtering.Firstly, we analyses the current research of recommendation engines, and introduce thecollaborative filtering technology in detail, and then introduce the current solutionmethods of data sparsity problem in collaborative filtering and their problems. To solvethe problem, the methods are proposed from two aspects in this paper: one is toimprove the existing method of similarity measurement in order to find a bettercollection of neighbors. This method based on measurement method of the Naive Bays,adding the popularity of projects as a penalty factor, The experiment data shows thatthe improved methods can be better reflect the similarities between "neighbors"; theother study emphasizes dimension reduction for singular value decomposition, whichmakes the computation of the original matrix more dense. An improved algorithm fordata sparsity is proposed, BAS algorithm: It firstly uses the traditional method ofsingular value decomposition to fill sparse scoring matrix, and obtains the forecastscore data, then uses these predictive score to obtain the neighbors of the active users,finally obtains the forecast data for the users by the method of adding penalty factor ofmeasurement method based on neighbor matrix. And analysis the correspondingtheoretical of the improved algorithm, experimental comparison between theconventional method of the traditional one on the selected data set. The experimentresults argue that the algorithm efficiently solutes the inaccurate problem ofrecommended precision with the data scarcity, to a certain extent, improves the qualityof recommendation engine recommends.