Virtual Computing Resource Managing And Scheduling System Developing For Networked Test

In networked test system, making use of computing resources within intelligent instruments for local data processing is increasingly becoming popular. Compared to the typical centralized mode in which all the data is processed by testing server, the localized processing method can achieve much better real-time feature for the test data processing procedure, sharply lessen the bandwidth occupation by reducing the amount of raw test data to transmit, and meanwhile lower the burden on the testing server host. In consideration of the distribution and heterogeneity characteristics of networked test system, it is essential to reasonably organize and efficiently manage all the intelligent instruments, and assign a task to the proper instrument according to the task’s characteristic and the real-time status of resources. Only in this way, can we fully utilize the limited computing resource of intelligent instruments to get work done cooperatively in premise of system stability and reliability. Based on that demand, this paper designed and developed a comple te computing resource managing and scheduling system.After a comprehensive survey of domestic and foreign well-known resource managing and scheduling schemes in grid computing and cloud computing area, this paper proposed the corresponding hierarchical system architecture for networked test, which divided the whole system into the following four parts in accordance with different functions: user layer, resource managing and scheduling layer, virtual resource layer and physical layer. Wherein the resource managing and scheduling layer is the core of the system, which is a connecting link between the preceding user layer and following resource layer, making both of them transparent to each other. The customers in user layer are able to query resource and submit tasks, while the virtual resource layer is mainly responsible for collecting and reporting resource information of intelligent instruments and dealing tasks in addition. Every layer communicates with neighboring layers efficiently and reliably by unified interfaces- Zero MQ message queue, which is an open source library for network communication.This paper abstracted the computing resource within all of the intelligent instruments in networked test system as a huge virtual resource pool to realize unified management. The managing system provides the following five functions. Firstly, it can describe resource such as CPU, memory and network bandwidth with a uniform expression form. Secondly, it can store the resource information to MySQL database and update it real-timely. Thirdly, it can sense the random entering or leaving action(either active or passive) of intelligent instruments and take related measures timely. Fourthly, it can locate the physical instrument based on its resource information. Finally, it provides a flexible resource query function.With global resource status information provided by managing system, this paper designed and developed a resource scheduling system. This system supports concurrent tasks requests, in which the scheduler is always ready to receive task requests from any user and added them to a task queue with a unified description. Scheduler selects tasks from task queue abiding by the FIFO(First In First Out) regulation, and then in accordance with feasibility and data locality principles it assigns tasks to the available resource nodes where the test data to be processed is located. Then the scheduler will real-timely monitor the states of the allocated task. In addition, this paper designed load balancing and fault recovery mechanisms to improve the system stability and reliability.To solve the contradiction of test data between dispersion in space location and centralization in processing logic, this paper designed a system for task decomposing and merging. According to the spatial distribution of data, decomposing module decomposes the integrated data processing tasks submitted by users into several sub-tasks, establishes the correspondence between the original tasks and sub-tasks, and further submits the sub-task set to the resource scheduling system to achieve task allocation. Merging module periodically queries about the execution status of the sub-tasks, and then merges all the sub-task results divided from the same original task to get the eventual result as soon as the last sub-task result of the original task arrived.The verification of physical platform proves that the designed resource managing and scheduling system has a satisfactory performance in perfect function, good expansibility, strong concurrency, high stability and efficiency. Compared to the centralized processing model, the designed system greatly improves the real-time performance of test data processing and resource utilization of intelligent instrument, reduces the pressure of network bandwidth, and decreases the workload of control mainframe.