Research On Task Scheduling Architecture For Many-core System In High Performance Computing

In the field of high-performance computers,how to allocate resources such as processor cores according to the task requirements submitted by users,and design flexible structures that support software definition and hardware reconfiguration are important research contents for computer architecture.But at present,there are still many issues in this field such as the dynamics of scheduling goals,the diversified constraints of tasks,and the diversity of task resource requirements,and so on.The above issues have become important factors restricting the development of high-performance computing technology and will affect the efficiency of many-core task scheduling.Therefore,based on the dynamics of scheduling targets,the diversity of tasks and resource requirements,a targeted task scheduling architecture that can support efficient multi-objective optimization scheduling algorithms and scheduling targets plug-in function will be proposed,which is conducive to the improvement of scheduling efficiency and system effectiveness.Based on this research content,the general paradigm research of the many-core task scheduling architecture has been done and the factor organization model(FOM)of many-core task scheduling architecture has been proposed.And on the basis of the FOM,an experimental platform sim Scheduler that could simulate different many-core task scheduling architecture and a Linux-based plug-in scheduler Arch have been designed and developed.The specific work undertook within these experimental platforms in this thesis is as following.To support efficient multi-object optimizing scheduling algorithm and realize flexible plug-in of scheduling objectives,the thesis proposes a more flexible,efficient,and extensible many-core task scheduling architecture: OLTSS(Optimistic-Lock and Transactional Shared State),which is able to add new policies and specialized implementations to the many-core scheduling architecture in the form of plug-in components.The task scheduling architecture allows tasks with different optimization targets,or displaying different application feature to adopt different resource scheduling strategies according to their own characteristics,and discloses all computing resource information to them in the shared state manner.Thereby OLTSS task scheduling architecture provides the way for tasks to free compete for computing resources in parallel to improve the utilization of resources.To solve the problem of resource competition caused by parallel execution of task scheduling in OLTSS many-core task scheduling architecture,the thesis proposes a resource competition coordination algorithm based on the optimistic lock.This algorithm takes advantage of the optimistic lock control mechanism when computing resource competition occurs between parallel task scheduling,coordinating and finally enabling different user-level schedulers acquiring computing resources concurrently in a freely competitive manner.This algorithm increases the concurrency performance of the task scheduling architecture and improves the node execution efficiency.Then,simScheduler is used to quantitatively analyze and compare different task scheduling architectures.The results have shown that the average task waiting time,workload completion time,scheduler usage and fairness of OLTSS task scheduling architecture with a resource competition coordination algorithm based on the optimistic lock are much better than that of other scheduling architectures.Subsequently,in order to fulfil the goals of users and system designers,a multi-objective optimization scheduling algorithm Multiple Object Fairness Tradeoff(MOFT for short)has been proposed that minimizes workload completion time and task waiting time while meeting storage capacity and constraints.With the OLTSS task scheduling architecture,the thesis simplifies the multi-obective optimization scheduling into two stages,consisting of conflict detection and conflict judgment based on different weight functions to improve the ability of resource allocation on demand and improve system performance.Then,scheduelr Arch is used to quantitatively analyze and compare different task scheduling architectures.The results have shown that considering specific resource allocation,satisfying storage constraints and guaranteeing fairness in a real experimental environment,the average task waiting time,workload completion time and fairness of OLTSS task scheduling architecture are much less than that of other scheduling architectures.Under the premise of ensuring the fairness of system resources,a conflict reduction control algorithm that minimizes the makespan has been proposed to solve the resource conflict surge issue that is caused by the rapid increase in the number of userlevel schedulers and tasks,and to realize the OLTSS's ability to accommodate more userlevel schedulers without reducing system performance and guaranteeing resource fairness,that is,plug-in capability,thereby improving its scalability.The conflict reduction control algorithm uses P control to dynamically adjust the system's plug-in capability to adapt to the floating parallel capability provided by the system,and uses a relative gain calculation formula to reduce the impact of the residual to achieve the goal of conflict reduction.Quantitative comparisons are made between the pluggability and scalability of OLTSS and OLTSS with conflict reduction.The results show that when the number of user-level schedulers increases rapidly,OLTSS with conflict reduction is performing better in the user-level scheduler usage rate and transaction conflict rate.It shows greater advantages and alleviates the effect of conflict surge.When the number of tasks increases rapidly,OLTSS with conflict reduction is less than OLTSS in terms of average task waiting time and makespan,reflecting the ability of conflict reduction control algorithms to improve the pluggability and scalability of OLTSS.Then,a quantitative comparison between the OLTSS with conflict reduction and the static partitioning in terms of makespan and resource utilization rate has been carried out,and the result once again reflects the superiority of the former.