The Design And Implementation Of Storage And Data Aware Scheduling Module In Grid Computing

Using the internet, we can form many computers into a―super computer‖, this method of computing is called grid computing. Grid computing can make the best use of the free resources in the internet, grid computing can also take full advantage of the rare resources in the internet, and grid computing can offer a very powerful data process service.This dissertation is based on a grid computing product called LSF developed by Platform computing corporation. LSF is good at managing computing resources such as CPU and memory, but LSF is not so good at managing jobs that produce much data or jobs that use much data. In this dissertation, we call jobs that produce much data producing jobs and jobs that use much data consuming jobs, and we call these two kinds of jobs storage and data aware jobs. The resource used by producing jobs is called storage, and the resource used by consuming jobs is called dataset. In order to manage storage and dataset effectively, designed and implemented storage and data aware jobs management module. In order to schedule storage and data aware jobs effectively, proposed and developed storage and data aware scheduling arithmetic, designed and implemented storage and data aware jobs scheduling module accordingly.The arithmetic solved the problem of the low efficiency of scheduling storage and data aware jobs and the problem of not enough disk space when storage and data aware jobs running, thus LSF can manage and schedule storage and data aware jobs more effectively and safely.This dissertation followed the current product's system architecture and scheduling framework. The scheduling module is developed as a plug-in, it also updated the code logic of other modules. The management module collected the disk data that storage and data aware jobs needed.Through this dissertation, I developed the whole module and successfully integrated the storage and data aware scheduling module into the LSF product. It raises the performance of the scheduler when allocates the resources reasonably.