Design And Implement Of The Comprehensive Job Hunting System Based On Big Data

With the continuous development of education and market economy,the scale,mode and market demand of job-seekers are changing rapidly.The traditional way of job-hunting requires that job-seekers pay attention to multiple job-hunting platforms at the same time,which significantly increases the cost of job-hunting.At the same time,the data of multiple job-hunting platforms can't be exchanged,and can't effectively provide aggregate search and personalized recommendation services for job-seekers,which affects job-hunting experience and thus reduces job-hunting efficiency,and is difficult to adapt to the development of job-seekers and talent market demand.In order to break down the job data barrier of job-hunting platform and make full use of job information of various job-hunting platforms,a comprehensive job-hunting system based on big data is designed and implemented.The system mainly integrates real-time job data collected from major job-hunting platforms,and uses Spark data platform to build a complete job data cleaning,storage and analysis process,thereby realizing job search and personalized job recommendation.Firstly,this paper analyses the job-hunting business process of the existing mainstream job-hunting platforms,finds the job data barriers of the existing multiple job-hunting platforms,and then puts forward the function of integrating job data across platforms to provide one-stop search and recommendation for job-seekers.Secondly,it describes in detail how to use Spark platform,integrate Scrapy to complete job data acquisition and cleaning,integrate ElasticSearch to achieve one-stop and fast search of job data,and combine K-means clustering method to cluster the acquired job data according to its TF-IDF value,so as to realize the relevant recommendation of job data.In addition,a combination recommendation method based on job content and job seeker collaborative filtering is used to provide personalized job recommendation service for job seekers according to job information and user's matching words,browsing,collecting and searching job log information.Finally,using data visualization technology,the characteristics of job information in different dimensions are demonstrated and analyzed in detail,which can provide a basis for decision-making of job seekers and other relevant users.The use of comprehensive job search system based on big data will provide job seekers with a job aggregation search and personalized recommendation platform to improve job search efficiency and job search experience.At the same time,through the multi-dimensional analysis of job data,it can provide professional analysis and advice for universities,educational institutions and other job market.