Design And Application On The Scalable Multi-dimensional Recommendation Engine Framework

With the development of Web2.0, information technology and the ubiquitousnetwork media like tablet PCs and smart phones, people have been from a lackinformation era towards the era of information explosion, in which there areoverwhelming message sprang up on the network and phone every day. Obviously, wehave entered the era of big data. Compared with the era lack of information, althoughthere are more choices, it is a great challenge for the information consumers to locatethe valuable and interesting information quickly from the information ocean. At thesame time, for the information producers, how to push the information to the rightconsumers and how to expand information radiating is not a simple thing,either. Inorder to alleviate the pressure of information overloading and enhance consumersatisfaction, personalization recommendation service came into being.Recommendation system establishes the user preferences model throughanalyzing the user behavior of history data, thus making relationship between usersand information. Recommendation system can push information to those users who areinterested in it, so user can get their information quickly from the information sea.The main contents of this project is to implement a multi-dimensional, flexibleand scalable recommendation engine framework. According to the methodology of therecommendation engine, recommendation engine includes three core components:recommendation algorithm, similarity calculation, recommendation post filter. Thispaper picked up collaborating filtering which is widely used in industry and academe;similarity calculation includes Jaccard Similarity, Cosine Similarity, Euclid DistanceSimilarity, Pearson Similarity for different scenario; recommended filtration mainlyachieves the user behavior filtering, item average filtering, location filteringrecommended these three filtering methods to increase the overall quality and accuracyof the results. Multiple dimensions of this topic is mainly reflected in the adoptive oftime sequence and spatial content dimensions based on the simple <user,item,rating>data model. Time sequence dimension is mainly uses the time decay function tobalance the similarity calculation and interesting calculation; the spatial dimensionmainly uses calculating the distance between items to achieve the recommendedfiltered. This recommendation engine framework implementations are based on the simpleand easy Hadoop Map Reduce distributed programming framework, and theunderlying file is stored in the Hadoop Distributed File System.This framework has been applied in flight reservation system for hotelrecommendation, based on the history of hotel booking information. In order to adaptto the requirement of the system, it adds the recommendation results analysis andstores the result in the database, thus the friendly integration with the flight reservationsystem.