Research On User-based Collaborative Filtering Recommendation Algorithm In The Case Of Sparse Data And Noise

With the further popularization of the Internet,especially mobile Internet and the arrival of big data age,people are increasingly troubled by information overload problems.The birth of the recommender system effectively addresses this problem,and it has been widely concerned and studied since being presented last century.Nowadays,different recommendation technologies have been widely used in various fields,which not only bring additional interests to the business,but also enhance experience of the Internet users.Collaborative filtering recommendation algorithm is one of the most widely and successfully used recommendation algorithms.Among them,the user-based collaborative filtering recommendation algorithm is to measure the similarity between users,and then select the neighbor set based on the similarity size,and finally recommend the target user according to the score of the neighbor-set users.This thesis studies the user-based collaborative filtering recommendation algorithm and the main works are as follows:Firstly,to deal with the data sparsity problem of the recommender system,a user similarity model combining Bhattacharyya coefficient and Jaccard distance for sparse data is proposed.This method not only uses the information of the co-rated items,but also the information of the non co-rated items,so it can better alleviate the problem of the data sparsity.In addition,this method uses Bhattacharyya coefficient and Jaccard distance to calculate global similarity between items.When the local similarity among user's score is calculated,the user's average score and the median of user's score is taken into account simultaneously and the Sigmoid function is combined to better calculate the similarity between the each pair of user's scores.Finally,the similarity information between the items is combined with the similarity information of the each pair of user's scores to calculate the final user similarity.Experimental results show that the proposed similarity measure can better select the neighbor-set users for the target user,thereby enhancing the final recommendation quality.Secondly,aiming at the problem that noise information is contained in the original data of the recommendation system,a method of detecting and removing the user's noisy scores is proposed.Firstly,the existing similarity index is used to calculate the similarity between the items.Then the similarity of the items is used to predict the score of one item rated by the user.Finally,the actual score of the item rated by the user is compared with the predicted one,and the former is determined as noisy score and is removed if the absolute value of the difference between the actual one and the predicted on is greater than the pre-setting threshold.After the above pretreatment,the traditional user-based collaborative filtering recommended method is used.The method not only integrates the similarity information between items into the user-based collaborative filtering recommendation algorithm in a certain way,but also detects and removes the noisy scores at the level of the user's scores,retaining other normal scores of the user.The experimental results show that this method can improve the final recommendation effect.Thirdly,to deal with the data sparsity problem of the recommender system and the problem that calculation of similarity between users only based on user's rating information will result in unsatisfying recommendation results,and taking into account the advantages of differential evolution algorithm,a recommendation method combining demographic information based on differential evolution algorithm is proposed.This method first calculates the user rating similarity factors with different emphasis,and quantifies the user's different demographic information to form different similarity factors related with corresponding demographic information.When the different feature information similarity factors are combined,the setting of the weights needs to be optimized by the differential evolution algorithm,so that the different feature information similarity factor shows the appropriate contribution in the final calculation of user similarity.The experimental results show that the optimized user similarity measure in combination with user demographic information can achieve better recommendation effect to some extent.