The Research Of File Scheduling Optimization Based On Cluster

Cluster is a group of computers or workstations connected in a particular way to provide users with a set of network resources and computing resources as a whole. Through the cluster system, you can get the super-parallel computing power and processing capacity. With the rapid development of distributed technology and parallel computing, the requirement for the cluster system is increasing. At the same time, it can provide powerful computing capabilities.But facing to the complex structure and a great quantity of nodes in cluster system,the system management must will be a very important field of study.The main content of this study is designed for Japan’s Fujitsu super-cluster system to bring about a file transfer scheduler. In condition of system stability, load-balancing of resources and timely running of the job, the scheduler can make sure the transmission time of file relied by job is as shorter as possible.This scheduler will provide a strong support for efficient job execution.Based on lots of research about popular solution on file translation in cluster environment currently and according to the actual cluster structure, this paper will bring up an staging transmission optimization algorithm.Then, I designed and implemented a simulation of the cluster system.The system contained the main management module, file staging transmission scheduling module and the API module. The main management module included the user input instructions, the nodes management, the jobs management, monitoring job’s state and scheduling procsesing of jobs.The API module included system controller and shared memory management functions. File staging transmission scheduling module implemented the file staging transmission management and scheduling. The optimization algorithm included the classic scheduling algorithms,the timeslice mechanism and getting the job’s request information timely.According to the comparing the value of available time slice on each nodes,the algorithm will choose the best shcedule transferring solution which can make sure the whole transferring time is the earliest. In addition, programming achieved the data communication between processes, and using mutex can protect data synchronization and security. Finally, after some typical user case testing, the whole calculation process is correct and the scheduling results meet to the desired objectives to achieve the shortest transfer time optimization goals.It can provide the best schedule decision support for the job’s efficiently execution.