Design And Implementation Of A Music Recommendation System Based On Streaming Big Data

With the development of mobile Internet and the popularity of smart devices,the amount of data generated by various websites and applications is increasing day by day,and the importance of data is also increasing.In order to make good use of these data,extract the user's interest and dig out the deeper value of the data,the recommendation system came into being.However,the traditional recommendation algorithm models,such as collaborative filtering and matrix decomposition,have poor performance in dealing with high-dimensional sparse data,and the generalization ability is weak.Moreover,it is difficult to ensure the real-time performance of recommendation,and the training cost of the model is high,it is difficult to directly apply it to production environments.To solve the above problems,a music recommendation system is designed and implemented based on streaming big data framework,approximate neighbor search technology and factorization machines.The system provides users with music information browsing,collecting,commenting,scoring and other business functions,and recommends the music they may be interested in.The system implements two different scenarios of recommendation function,which are similar content recommendation and user personalized recommendation.The system combines B/S architecture and streaming big data framework technology,collects user behavior data through Java server code burial point,stores user behavior data using message queue Kafka,then calculates and transmits user behavior data in real time by using Flink.The recommendation algorithm introduces the idea of word vectorization in natural language processing,combines TF-IDF algorithm to vectorize the text information of music,and uses redis database to store music vector;realizes locality sensitive hashing algorithm through random projection method,which is applied to similar content recommendation and recall stage of personalized recommendation.Finally,factorization machines algorithm is used for training the user-music behavior characteristic data,calculate the user personalized recommendation result set,and push the recommendation results to different users.Music recommendation system has good scalability and maintainability.The system realizes the locality sensitive hashing algorithm based on the random projection method,achieves the purpose of fast approximate neighbor search,and enhances the real-time performance of recommendation by combining with the streaming big data framework,achieves the recommendation requirements of two different scenarios,and improves the user experience.