Design And Implementation Of Generic Web Based Small-scale Cluster System

With the rapid development of information technology, it plays an increasing role in scientific research. Espacially for the highly complex computing issues in the theoretical analysis and scientific experiments, high perfomance computing is becoming a helpful and efficient tool. Large workstations, servers and clusters are widely used, and in which clusters have a better high cost performance ratio. Howerver, there exist many small-scale high permformance scientific computing scenes in research work, where the task logic requires high computing resources but is relatively simple and with more types of task and higher frequency. Meanwhile, multi-role collaboration features is needed. Conventional cluster cannot meet in such research scenes. Therefore, considering both high performance with low cost and good extensibility, a lightweight cluster job management system becomes necessary.To satisfy the requirements in small-scale high performance computing issues, we designed and implemented a lightweight generic cluster job management system based on web.Firstly, the paper puts a detailed analysis of scenario and requirements for the small-scale high performance computing issues commonly found in many research work. The core features include task management, cluster status monitoring, user management and others. Then, it explains the overall framework from three aspects, including the physical deployment, the logical hierarchy and the executing flow. The key points and difficulty are analyzed. We proposed a concept of task-template, by using template, the system can support new types of task easily. This will reduce much redundancy work, and provide a better extensibility. Besides, it makes a introduction on the data storage, task scheduling, transmission middleware and web service. With charts and key snippet, it describes the design and implementation of core modules, and shows the final front-end screenshots. Finally, the system is tested under the real the real deployment environment, those result indicates that the system serves well as expected, the user interface and interaction is user friendly.This generic web-based cluster job management system provides a lightweight and low-cost solution for many small-scale scenarios of scientific computing in research. Withtask-template feature, one can quickly create tasks of a new type. A reasonable scheduling and management makes cluster with multiple machines maximize its computing ability. Web-based architecture brings better usability and better support of the maintenance and upgrading. Therefore it can be widely applied in theoretical analysis and scientific experiments.