Design And Implementation Of Bidding Information Automatic Capture System Based On Heuristic Search Algorithm

In recent years,the number of bidding data such as engineering bidding,centralized procurement,service bidding and so on keeps increasing,and more and more enterprises seize the market by bidding.However,at present,many enterprises still prefer the traditional manual way to obtain bidding information,the main reason is that the bidding data is relatively scattered,and presents the characteristics of regionalization.However,the amount of bidding information is huge.If we only rely on manual means,it will be difficult to collect,and it is very likely to miss the important bidding information.Therefore,how to collect the data in the field of bidding and integrate the information resources has become the main research hotspot.For some bidding websites with anti-crawler mechanism,these websites will replace the link address in the webpage randomly on a regular basis,so that people can't obtain the data directly by accessing the URL link.Due to domain name filing and other reasons,the official website of each bidding website will not change basically.How to use the crawler program to link from the official website and quickly find the URL link of the target page will be the focus of this thesis.For these special websites,traditional crawler programs generally use the breadth-first search algorithm,which blindly searches the data without taking advantage of the characteristics of the data to be crawled.The more websites from which we need to crawl data,the more time we will take.Therefore,the author designs and improves an adaptive heuristic search algorithm.According to the relevance between the page and the topic as well as the related information of the link,it gives priority to select the link with higher possibility to crawl,eliminates the irrelevant link,reduces the blind search,and finally modifies the heuristic function according to the crawling results.The experimental results show that the time of crawler using heuristic search algorithm is significantly less than that of crawler using breadth first search algorithm.Based on the above reasons,the author designs and implements a set of bidding information automatic capture system based on heuristic search algorithm.The system is mainly composed of user login,crawler module,bidding data search,bidding data display,subscription management and system management.The automatic capture system of bidding information regularly and automatically obtains bidding information from 154 bidding websites every day,saving a lot of labor time,and notifying users of the content they are interested in by e-mail.The system has beautiful interface and powerful function,and can run well after testing.