The Design And Implement Of Search Engine System On Retrievaling Recruitment Information Use Scrapy-based Framework

With the rapid development of the Internet,Internet users can search a lot of information on the internet easily with the help of search engines.So,how to establish a connection and access to information is no longer a problem plagued users.In the field of job recruitment,the way of job hunting and recruitment through the Internet is increasingly becoming the first choice.Because of involving almost all professional fields,traditional recruitment website is difficult to collect enough information.At the same time,because of differences between different professional fields,it cannot make a deep analysis of the obtaining recruitment information,so it cannot provide users with recruitment information analysis services with reference value.In the new age of the Internet,job seekers are no longer inclined to pay for such a large and rough way of information processing,otherwise,the job seekers urgently need a search engine based on the specific professional field of recruitment information.The design and development of the recruitment information retrieval system is essentially a recruitment information search engine,which is based on the field of computer and software engineering professional segment,aiming to provide talents in software engineering field with excellent and detailed service.In the process of system design and development,firstly,based on the analysis of several recruitment websites,we collect user's requirements,model of the system,and establish the functional requirements of the system.Secondly,we study a lot of the technology and literature relating to information retrieval systems,and give the architecture design of the system.We decompose the functional requirements of the established system and develop the functional modules for each segment.Finally,we integrate the completed functional modules and test the integrated system.The main functional modules of the information retrieval system are: user registration and user login,retrieve recruitment information,visualize recruitment information analysis,enterprise user release recruitment information,collect recruitment information,format and storage recruitment information,manage registered users,manage recruitment information and so on.In the process of constructing the recruitment information retrieval system,we develop a number of crawler-specific crawler programs using the Spider module in the Scrapy Open Source Reptile Framework,and these crawler-specific crawler programs collects information from the recruitment web site.We format the recruitment information collected by the crawler program through using the Item Pipeline module in the Scrapy framework.And through the restful interface provided by Elasticsearch,the formatted information is persisted to the Elasticsearch Search server cluster.To provide easy-to-use retrieval methods for job-seekers,we use Yii Open source PHP framework to develop business-side processing modules to handle user's requests from the Web browser side.The significance of design and develop the recruitment information retrieval system is to provide a more comprehensive information retrieval system for the computer and software engineering talents by integrating information on multiple recruitment information websites.The test results of the system show that the system can meet the functional requirements of the talents in the field of computer and software engineering.At the same time,the service of recruitment information analysis provided by the system can provide effective guidance to achieve the desired objectives.