Design And Implementation Of Bidding Information Acquisition And Analysis System

At present,China's network technology is developing rapidly,and more and more companies choose to publish their bidding information through the network platform.Network bidding not only has the advantages of fairness,justness,and openness,but also enables efficient bidding and greatly reduces transaction costs.For each bidding company,every piece of bidding information on the network platform is extremely important,and if it is missed,it may cause them huge losses.However,the widespread application of online bidding has also revealed that they have problems such as incomplete collection,analysis errors,and time lags in the collection and analysis of bidding information.There are three main reasons: first,information dispersion,second,information The number is huge.Third,the timeliness is insufficient.In addition,the bidding website failed to make full use of technologies such as text analysis in the era of big data to discover useful information.Under these circumstances,bidding websites that rely on manual collection and analysis have been unable to adapt to the increasingly fierce market environment and urgently require automatic collection and analysis of bidding websites.In response to the above problems,this paper designs and implements the collection and analysis of bidding network information based on a variety of technologies such as web crawlers and topic models,OLAP,and optimizes the system by adding data buffers and establishing indexes in the database.Users provide a platform that can quickly and efficiently obtain the required bidding information.The main work of this article is as follows:(1)Automatic collection of bidding information.Crawls based on keywords and URLs set by users,regularly obtains information from major bidding information publishing websites,divides the data into different dimensions,levels,and stores them in the database in a star schema.By establishing indexes and analyzing data distribution methods,The database is optimized,and at the same time,a data buffer is added to the system to improve the response efficiency of the system,so that the collection and analysis of bidding information is performed efficiently.(2)Analysis of bidding information.The TOT model and the LDA model are constructed using Gibbs sampling,calculation of confusion,and so on,and the bidding data is analyzed from multiple dimensions through conventional statistical analysis methods.Finally,conclusions are drawn on the evolution trend of the bidding industry categories in time,the main bidding industries in various regions,and the distribution of bidding projects in the region.These analysis results can help users to see the changes in the bidding industry on the one hand,but also help users understand the main needs of the current bidding market,and help companies to formulate more accurate business strategies and development strategies.(3)Visual display.Combines visualization tools such as Echarts and Matplotlib to visualize the results of bidding information analysis,and displays them in an intuitive way,enabling users to quickly understand the trend and development trend of the bidding market without reading a lot of text,and real-time performance of all bidding projects And overall improvement.The implementation of this system can help users quickly obtain bidding information,accurately locate the market direction and trend,greatly reduce the burden on users,and has high practical value.