Design And Implementation Of Micro-Blog Recommendation System Based On Hadoop

Micro-blog is a huge social network containing a huge user group and vast amount of data, on average, about20million micro-blog items published everyday and the number of user is still growing. User would feel very confused when facing this mass information, but their using experience could be better if system can be set to recommend people or news they may concern, helping them to build up their social circle quickly, and getting better access to information. This thesis research how to design and implementation a micro-blog recommandation system based on hadoop. The main work is as follow:First analyzed the progress of the micro-blog recommendation system, and analyzed related technology, including Hadoop distributed framework, NoSQL database, word segmentation technolgy, etc.Secondary, In the light of Sina-Weibo, Starting from data processing, analysis, storage and display. A design of framework for micro-blog recommendation system was given, which include capture module, analyzes module and display module three subsystems. The capture module includes crawlers, ETL software and MongoDB; The analyzes module include key-word recommendation, user recommendation and weibo recommendation; The display module include user management, key-word recommendation, user recommendation and weibo recommendation. As well as the design of table. Based on platform of hadoop, desgin and implement all modules of above.At last build a platform of test system for testing and verifying the performance of system, the ability of data grabing and analyzies, the result of weibo recommend. With these testing cases we can prove the system that can reach the target which it is designed to be.The entire recommendation system applies the distributed architecture design, with characters of high reliability, scalability and strong computing ability, is ideal to process the massive offline micro-blog data rapidly. The application of the system is able to help user easily viewing Sina Weibo, to increase the attachment between user and system, and can also provide accurate and timely data support to enterprises or research work.