Research And Implementation Of The Downloader Supporting Multiple Protocols And Multiple Threads Based On Linux

Network file download is always an important force to promote the Internet develope. People have designed many protocols and methods to make file exchange fast, secure and efficient. The methods for file download through the following stages: multithread file download based on HTTP and FTP, multithread file download based on P2P protocols with the features of multisource data transfer, multithread and multisource file download supporting various network protocols.Download tools on Linux environment have high download speed and transplant easily, but all of them are poor in function integrity. For example, command line is not as convenient as Windows, the limited category of protocols are supported and someother defects. Based on the study of HTTP,FTP and BitTorrent procotol, the content and constributions of this thesis are as follows:Firstly, compared with various download tools on Linux, this thesis finds defects and advantages of them and makes a brief analysis of HTTP, FTP and BitTorrent in principle.Secondly, compared with random neighbor selection strategy, this thesis improves neighbor selection algorithm. Our experiments provides evidence that this algorithm not only makes the best use of bandwidth between peers that in the same neighborset, but also ensures the probability of getting new file pieces from neighbor peers. So, it improves the efficiency of the whole system.Thirdly, contraposing the useless of incentive mechanism in the circumstance of a large number of seeds, this thesis raises a bandwidth allocation strategy for seeds based on the research of incentive mechanism of BitTorrent-like p2p file sharing system. Our examination shows that this mechanism can optimize the allocation of seeds'bandwidth, effectively avoid the production of free rider, reduce the average download time of contributor and improve the impartiality of the whole system.Fourthly, contraposing tracker's heavy load which arises from peers'joining or leaving frequently and bandwidth loss caused by random neighbor selection strategy, this paper raises a file sharing system based on neighbor cluster model and declares the application of super peers selection strategy, peers join strategy and peers offline strategy. Fifthly, combined with improvements mention above, this thesis constructs a download system by features of supporting HTTP, FTP, BitTorrent and multithread. In addtion, it elaborates on block files, multithread synchronization and breakpoint resuming for HTTP/FTP download mode. Then, it lists the BitTorrent download mode which emphasis on seeds analysis module, strategy selection module and peers management module.In system testing, compared with download time, the average download speed and memory usage of wget, d4x, lftp, bittorrent and someother download tools on linux, it provides evidence that this download tools can be stable during the download time with high download speed. Compared with the average download time in the simulation experiment, it showed that the feasibility of algorithms and system models.