Research On Spark-based API Recommendation System

As the society continues to advance,the amount of Internet information has grown exponentially,and it has become increasingly difficult for users to access information.As a new lightweight Internet application service in recent years,Web API services can build services for various clients,and can be freely reused and combined to make it easier for users to use various functions.As Web API services are gradually being used by the public,how to help users find services that match their needs more quickly becomes difficult.Recommended system appears to solve this problem,but the use of traditional recommendation systems in the case of the amount of data and number of users Web API services increasing,the recommendation system will create enormous pressure.Spark-based computing framework is widely used in the processing of massive data,and provides new opportunities and challenges for the research of massive Web API service recommendation system.The main content of this paper is to personalize recommendations for Web API services,the method is to solve the problem that the final result is not accurate enough by solving the sparse data in the traditional algorithm,and the result of the classification of the original data is not standard,improve the accuracy of the Web-oriented API recommendation system by improving the similarity calculation,at the same time,combined with the Spark computing framework,it is better adapted to the processing of massive Web API data and improve the execution efficiency of the recommendation system.The key research aspects of this paper are as follows:(1)This paper focuses on two improved algorithms,including an improved collaborative filtering algorithm and an improved content-based recommendation algorithm.The improved collaborative filtering algorithm solves the problem of data sparsity by improved null filling method and mean centralization method,and then introduces the trust degree and attention degree between users to improve the accuracy of recommendation results by calculating similarity.The improved content-based recommendation algorithm re-classifies the data to solve the inaccuracy of the original data by introducing the Fasttext algorithm,and then introduces the TFIDF algorithm to extract the text keywords to improve the accuracy of the computational similarity.(2)Parallelize the two algorithms in the Spark computing platform,introduce theparallelization process in detail,and finally show the comparison results of the parallelized experiments.The results show that the parallelized algorithm has greatly improved its running speed,so the personalized recommendation effect for Web API services can better meet the needs of users.Finally,this paper analyzes and designs the recommendation system of Web API service,including requirements analysis,system architecture and algorithm implementation.