The Design And Implementation Of Intelligent Computing System Oriented For High-throughput Biological Information Analysis

With the continuous progress of biological high-throughput sequencing technologies, the explosive growth of genomic data has brought huge challenge to the genomic data processing. To meet the efficient use of resources on high-throughput biological analysis processing program can not only rely on traditional common cluster scheduling strategy. Unreasonable user parameters and scheduling strategy not only increase of the task waiting time but also reduces the utilization efficiency of system resources. Therefore, research on design and implementation of an intelligent computing system that oriented for high-throughput biological information analysis is meaningful.This paper analyzed the system log of high-throughput biological computing system from BGI. Then design and construct an intelligent computing system that oriented for high-throughput biological information analysis based on task modeling for biological computing tasks. The system can model the historical task log, and then provide reasonable queue partition scheme and resource allocation scheme according to the modeling result. Meanwhile, simulation will be made periodically using different scheduling strategy in order to find the best one for the administrator. Therefore, scheduling strategy can adapt to the change of the system in order to improve the performance of the system and the resource utilization.This paper preprocessed the task log of high-throughput computing system and converse the format of the log. In addition, the resource usage of tasks, delivery characteristics of tasks and queue characteristics are also analyzed. Clustering analysis was made for the resource usage of tasks according to the log. After that, this paper made a simple division for tasks according to the resource usages of tasks. And then model the tasks by the quantification indicators at last.On the basis task analysis result, an intelligent computing system that oriented to biological information analysis was designed. This system support task modeling and intelligent scheduling strategy. On the one hand, queue division and resource allocation can be decided according to the task model, so that different kind of resource can be used efficiently. On the other hand, simulation can be made with different scheduling strategy in order to find the best one for the system. This paper proposed an optimized multiple queue scheduling strategy based on task model, simulation experiment for the new strategy is also done.The intelligent system provides the researchers a research platform for the study of scheduling and resource allocation. The standard interface provides by the system make it easy to add new scheduling strategy into the platform, then researcher can easily deploy, test and compare the new strategy, which improve the scalability for the system.