The Design And Implementation Of Job Recommendation System Based On Hadoop

In recent years, personalized recommendation system has brought a lot of convenience to people’s life and work, thus it has become the focuses of many researchers. However, with the explosive growth of information in network recruitment, simple job searching may be difficult to meet the needs of job seekers. Although more and more job recommendation technologies have been widely used and made popular, many recruitment websites do not analyze and explore historical behavior of jobseekers.This paper made an intensive analysis of job recommendation system development based on the behavior of jobseekers. Facing the issues existed in the traditional job searching and browse, a method analyzing the Web behavior log of jobseekers is proposed. Using recommending technologies of content-based filtering, population statistics filtering and collaborative filtering, a personalized job recommendation system is designed and implemented based on behavior analysis.In this paper, the main contents and research work are as follows:(1)The recommendation technologies of content-based filtering, population statistics filtering and collaborative filtering are applied in a personalized recommendation system. Job seekers behavior is divided into registration, browsing and application through the analysis of the applicant’s Web log. Seekers-jobs model is built. Through on-line real-time behavior data collection, using off-line data analysis and online real-time computation combined with summary and intention information of jobseekers, a comprehensive real-time referral service is provided for job seekers. The recommended service in the paper includes job-seekers-login triggered and successful-applicants triggered job recommendation. The development quality of the recommendation system is guaranteed by constraint of the system’s precision and recall rate.(2)The job seekers login-triggered recommendation service is realized using content-based filtering and demographic-based collaborative filtering technologies. Recommendation results are optimized by balancing the advantages and disadvantages of each technique. Jobs are filtered according to the matching degree of application intention and job content. Locally weighted similarity method is used to calculate similarity between seekers and jobs, and then seekers-job hierarchical model is built to determine the weight coefficient of locally weighted similarity in order to make the recommended jobs more interpretable.(3)The collaborative filtering technology is used to achieve recommendation service when seekers successfully apply for jobs. Job seekers-position feature vector combination is constructed including interest of seekers, freshness and similarity of jobs. Jobs are filtered according to the position id of seekers and the threshold of position similarity. Positive and negative samples are collected. Optimal weight is performed by machine learning. More accurate position recommending is gained by providing recommendation service for seekers through calculating seekers-jobs logistic regression scores.(4)In this paper, data storage, key algorithms and seekers-job model is achieved based on big data platform Hadoop. Online recommendation service is based on distributed platform Storm with stream computing ability, which can not only support storage and computing of massive data, but also improve the response speed of online recommendation. Meanwhile, data collection and transmission is realized based on log collection system Flume and message caching system Kafka to imply the job recommendation system.Finally, the function and performance of the job recommendation system proposed in this paper is evaluated to valid its practical application value.