| With the advances in network bandwidth, computational power, memory capabilities, and storage technologies, computational science has been evolving over the past few years towards data intensive computing. In contrast to this evolvement, TCP -- Transmission Control Protocol remains to be the most commonly used protocol for data-transfer, despite it being unsuitable for moving large volumes of data sets across the networks, particularly for wide area networks (WANs). This is due to the default TCP settings on most hosts that are configured to deliver reasonable data-transfer performance -instead of optimal performance- both on Ethernet local area networks (LANs) and on WANs. Therefore, in order to circumvent the performance drawbacks over wide area high-speed networks originating from a window-based congestion control mechanism of TCP and its default settings, proposals of different solutions have arisen over the years. However, most of these solutions are based on a client/server paradigm, and therefore are focused on improving the performance of data transmission between sender and receiver. When there are multiple receivers interested in the same data sets -a situation that is very common in scientific computing- this approach fails to ameliorate the performance of bulk data-transfer between the receivers.;In this dissertation, we present a novel GridTorrent architecture that is not only built on a peer-to-peer network model, but also combines collaboration and service-oriented computing principles with adequate security features such as authentication, authorization, and data integrity in order to provide an efficient, scalable, secure, and modular framework for high-performance data-transfer in scientific computing. Our lightweight architecture not only performs well on very high-performance networks but also on networks with limited bandwidth capacity. In addition to being deployable on any type of platform, its data transmission layer is a generic transfer layer that is independent of data type and format. |
