Network Middleware Data Acquisition System

The information industry represented of the computer in 20th century signifies human society entered the information age. The rapid development of network technology makes an important feature of this age; all kinds of information networks are widely used. The local area network users also need access to large amount of information from the Internet, for the organization and decision-making support. But the State Security Bureau requests computer information systems involving state secrets can't directly connect with the Internet or other public information networks between which must be isolated. Thus, we propose an information acquisition system design project based on the middleware . We establish an indirect connection between the Internet and local area network through one or more intermediate servers to ensure that any time LAN and Internet are isolated. At the same time, we rewrite the popular Wget download engine algorithm nowadays because of the low efficiency original information acquisition system. We realize the operation of information collection, information transmission and creation of a virtual site through establishing modules such as the task queue management, the collection device queue management, the attempter coordination management, the interior server management with information acquisition and so on. We generalize the system in our respective units after for Internet information collected in the local area.