The Study Of Collective I/O And The Optimization Of Data Reallocate Algorithm

An integrated research on collective I/O technology is made, and a further explored measures to advance the access to non-continuous data is discussed in this essay. In many parallel applications, each process needs to access of data with non-continuous position, and the data which are stored in the document. Numerous I/O requests would be necessary in order to obtain access to this kind of non-continuous fragments of data, which would definitely undermine the performance of parallel I/O. Therefore, I/O operation has already become the bottleneck for the efficiency of parallel applications. How to organize, store and efficiently access data, turns into a problem crying for research.Based on the two-phase I/O, a novel improvement solution is presented for the algorithm of data reallocation in the two-phase I/O: statistic-executive I/O, which optimizes the I/O of the most familiar cyclic-cyclic and block-block data distribution. We divide the parallel I/O into two phases: statistic and executive. Communication modes are calculated and automatically build into needed data types during statistic phase. This information is used during the executor stage in performing the communication and file accesses. The two phases are decoupled. Therefore, in repeated file access patterns, the computations from statistic phase could be performed once and reused several times during executive phase. This strategy allows to amortize the statistic cost over several I/O operations. In this thesis, we evaluate the performance of multiple phase I/O collective technique and we compare it with other state of the art approaches. Experimental results show that for cyclic-cyclic and block-block data distribution, our method outperforms in the large majority of two-phase I/O optimizations techniques.