Data management support for distributed data mining of large datasets over high speed wide area networks

Distributed applications are unable to utilize the bandwidth provided by high-speed wide area networks without extensive network tuning. Two new software libraries called Parallel Sockets (PSockets) and Selective Available Bandwidth Utilization Library (SABUL) have been developed to help such applications use the maximum available bandwidth efficiently.; Performance analysis of the PSockets and SABUL libraries were conducted on Abilene networks. Experimental results have shown that the PSockets and SABUL libraries provide the maximum throughput to applications with minimal network tuning. Detailed analysis of TCP while using PSockets has also been conducted in this thesis. Three new benchmarks are proposed to compare software libraries that are used to transfer data on high-speed wide area networks.; Using the libraries PSockets and SABUL the Data Space Transfer Protocol (DSTP), used for exchanging large data sets across the web, was extended for high performance clients. A new high performance DSTP server and client libraries were built with the two software libraries PSockets and SABUL. The high performance DSTP server serves large data sets efficiently over high-speed wide area networks and is used by distributed data mining applications being developed at National Center for Data Mining, University of Illinois at Chicago, Chicago, Illinois.