Design And Implementation Of Merchant Recommendation System Based On Spark

With the development of information technology,the Internet has brought a lot of convenience to People’s Daily life.People can browse the information in the website anytime and anywhere through the Internet.It brings us convenience at the same time,but also brings a huge amount of data.In the face of massive data information,users cannot quickly and accurately locate the information they are interested in,resulting in data redundancy.Based on this background,this paper designs and implements a merchant recommendation system based on Spark to help users filter out the merchant information they are interested in and solve the problem of data redundancy caused by massive data.The whole system uses MVC three-tier architecture design mode,and the development framework is Spring Boot to complete the development of Web application.Spring Boot’s internal integrated Spring MVC framework is used to control and forward the front and rear ends.My Batis maps with the main business database to save and read the recommended results,and the recommended results are finally presented to the system users through the front-end page.The core of the system is the recommendation function,which is developed and implemented by Spark distributed computing framework.Two recommendation functions,offline and real-time,are designed and implemented in the system.The offline recommendation function includes three modules: contentbased offline recommendation,cryptic meaning model-based offline recommendation,and statistics-based popular content recommendation.The content-based offline recommendation module builds users and item portraits for recommendation by calculating the weight value of label information.Because the item information in the system is more stable than the user information,the content-based recommendation module is very suitable for the cold start-up period of the system.The off-line recommendation module based on the cryptic meaning model combined with the user behavior matrix to predict the score.Different from content-based recommendation alone,this module can use the scoring information to reflect the quality of the articles laterally,and the recommendation result is better than the content-based offline recommendation module.The offline recommendation module of popular content based on statistics configuring Mongo DB connection information,connecting to the main business database and obtaining data information in the database.Based on quantity statistics,this module directly queries data in the database to get recommendation results.The recommendation results obtained have low overlap with the user’s historical behavior data,which can mine more user information and enrich user portraits.The real-time recommendation function includes two modules: real-time recommendation based on nearly k ratings and real-time recommendation based on content.In the real-time recommendation module based on nearly k ratings,a recommendation priority calculation algorithm combining rating and similarity is implemented.The module directly uses the similarity information produced by the offline recommendation module based on the implicit meaning model to calculate,greatly saving the running time.The content-based real-time recommendation module obtains the similarity information produced in the content-based offline recommendation module for recommendation,which meets the content-based recommendation in realtime scenarios.The offline and real-time recommendation functions are based on the hybrid recommendation mechanism and can be applied to different recommendation scenarios.At present,the system has passed all the test indicators and can meet the needs of system users.