Research And Application Of Collaborative Filtering Algorithm Based On Keyword Extraction Technology

The dramatic development of the Internet has also brought about the dramatic development of e-commerce,and the automated recommended items have brought good returns to major e-commerce websites.However,the rare item evaluation data raises a serious problem when facing a large number of recommended objects-the evaluation matrix sparsity problem.The excessive sparsity of the evaluation matrix seriously affects the accuracy of the similarity calculation between the objects to be recommended,such as the similarity calculation between movies.The film review of the film review site often summarizes the characteristics of the film.The keyword extraction algorithm can be used to extract features from the film review text to calculate the similarity between movies.Before the chinese keyword extraction,we should first use the appropriate word segmentation algorithm for word segmentation,and then select the appropriate keyword extraction algorithm according to the application scenario.In this paper,the keyword is extracted by evaluating the text of the captured movie,and then the similarity calculation of the movie is assisted by the keyword of the article.The experimental results show that the algorithm can effectively improve the recommendation quality of movies.However,there is still much space for improvement in the improved algorithm above.First of all,the description of the film will have certain differences in different types of movies.The idea of information gain can be used to weight the distribution of different vocabularies among different types of movies.At the same time,the word vector technology can be introduced to map all words into the word vector space,and then the word meaning comparison.Experiments show that the improved idea ultimately improves the recommendation quality and proves the feasibility of improving the collaborative filtering algorithm.Finally,this paper designs a movie browsing website that includes the recommendation system,and introduces some tasks with high throughput and delay requirements into Spark technology,and successfully implements the algorithm and practical application fields.