Design And Implementation Of Biomedical Compute Platform CSF-OPAL

Currently, large-scale scientific application compute platform is transferred from tradition grid way to high performance platform.The research goal of grid scheduler is to achieve cross-domain resources sharing. But with the characteristics of distribution, heterogeneous and autonomous, grid makes some limitations on sharing cross-domain resources and only the co-allocation of resources is hot. While, In cloud computing, computing resources and data resources are consolidated and integrated together. Users can share resources with service unit; And with the advantage of virtualization. we can achieve effective isolation between software and hardware systems and solve the problem of heterogeneous in grid platform. Obviously, the advantages of cloud environment are clearly, so it's very significant to research resources sharing cross-cloud platform when the scale of biomedical application need more and more resources.In the Software OPAL which supports scientific applications running, it deployed different scheduler as its JobManager, including SGE, Condor and many other local scheduler, Of course, including meta-scheduler CSF too. These schedulers have solved the problem which scientific application become bigger, However, the expand of single OPAL platform is limited, And there is need for service resource sharing among many OPAL platforms. So this paper study CSF and focus on two issues:1,Cross-domain resources management and use. For the problem of resources sharing among many OPAL platforms, we propose a resource sharing model consists of CSF and OPAL. In this model, CSF is in the upper layer as a scheduler and OPAL is a resource in the lower layer. The integration of CSF and OPAL is completed by adding a resource manager in CSF. We get resources using static way which is existed in CSF, then provide these resources for resource manager and finally is used by scheduler framework.Achievement and scheduling for array Job. For large-scale scientific application, we first extract its useful information and do rational analysis and build the common job model in CSF; Then we parse job description file and put resource requests into job data. Finally, we specify a feasible scheduling strategy and implement in CSF as plug-in.2,In this paper, we achieve the design of CSF-OPAL platform by implementing plug-in and resource manager mechanism. The resource manager makes it possible for resource sharing among many OPAL platforms. Using the communicate principle of web service, we implement job submit, status query and job control function. The new plug-in is used to ensure co-scheduling among OPALs and give the method of splitting and matching array job according to cpu number. The cross-domain resource sharing and co-scheduling of OPAL clouds is the focus of this article. This paper builds and implements a CSF-OPAL cloud model with CSF as research object and OPAL platform as resources vector. With biomedical applications as our experiment cases, we study and solve the problem of resource sharing and co-scheduling among several OPALs. In this paper, we add resource management mechanism in CSF to dispatch jobs to different OPAL cloud platform to achieve resource sharing among many OPAL clouds; then we add scheduling mechanism for new type job. In the context of new model, we achieve the objective of resource sharing and co-scheduling among platforms.