Real-time Resource Prediction System In Computational Grid Environment

As a latest emerging network infrastructure, Grid is the key technologyof the next generation of the Internet. Along with the huge improvement andrapid progress of computer performance and network communicatingtechnology, application demands are developing towards large-scale , high-performance, diversity and multi-function, so it needs to connect the varioushigh-performance computing resources, storage, data and other specificresources, which are geographically distributed and heterogeneous, via highspeed network to establish collaborate high-performance computing andsolve huge application problems cooperatively. It is called wide-areahigh-performance meta computing technology, Grid Computing Technologyas well.Grid can solve large-scale problems of science, project and business bychoosing, sharing and aggregating the resources that are geographicallydistributed and heterogeneous. The users with different resource demandswill share the dynamic and heterogeneous resources in grid settings. In adynamic grid environment, the problems within the system, such as resourceconfiguration, user competing, node failure and network congestion, havehuge effect on application performance, and it is difficult to predict a prioritynumber based on these factors, especially when it is constrained to access tosystem data. In addition, the owners of resources provide dynamic variedservices with collision demands. It is very important for load managementsystem to choose appropriate resources from dynamic varied resource poolfor applications. As a result, scheduling system can make decisions in gridsettings based on those real-time, accurate and dynamic predictedinformation of resource performance. It makes use of Grid resources morereasonably, guarantees threshold of task completion time and improves theperformance of application.Performance prediction and evaluation are both critical components ofthe Computational Grid architectural paradigm. In particular, predictions(especially those made at run time) of available resource performance levelscan be used to implement effective application scheduling. Because Gridresources (the computers, networks, and storage systems that make up a Grid)differ widely in the performance they can deliver to any given application,and because performance fluctuates dynamically due to contention bycompeting applications, schedulers (human or automatic) must be able topredict the deliverable performance that an application will be able to obtainwhen it eventually runs. Based on these predictions, the scheduler can choosethe combination of resources from the available resource pool that isexpected to maximize performance for the application.With the analysis and research of domestic and overseas predictiontechnology, we found that compared to application-oriented approach,resource-oriented approach provides information independent of applications.That makes it possible to make up for the defect between resource predictionand task performance, and makes scheduler based on explicitresource-oriented prediction approach more effective. So we make explicitresource-oriented prediction with resource signal method.Resource prediction system should have such characteristics as designwith extensibility, low overhead and providing accurate, real-time predictedresource performance. In Computing Grid environment, the development ofprediction system with such characteristics brings practical sense.In this paper, with the analysis of current achievements we adoptappropriate part of them, make necessary integration, extension, andimprovement, and set up a real-time prediction system in grid settings. Basedon the research of fundamental prediction mechanism and models, weeventually design and implement a real-time resource prediction system inGrid environment with the Network Weather Service and the MetacomputingDirectory Service. First, we enrich the prediction functions of NWS(Network Weather Service) with the introduction of load average prediction.It makes task performance prediction system and scheduler work moreeffectively. Second, we can dynamically choose prediction models to predictresource performance in order to obtain more accurate result. Third, resourceprediction algorithm based on computationally inexpensive statisticaltechniques supports adaptive application scheduling in Grid settings, andon-line fault detection. At last, we apply the system in Grid settings by theinformation service provided by MDS. It makes up for the defect that thecurrent prediction systems provide only local resource performanceinformation which can not meet the demands of users to operate transparentlyin Grid settings. The users and other applications in dynamic Gridenvironment can conveniently obtain the real-time, accurate, predictedresource performance, make reasonable use of grid resources and improveapplication performance.With unified data models, we have integrated real-time resourceprediction service based on NWS and MDS standing upon LDAP hierarchicalstructure. Based on analysis and practice of the MDS mechanism, weencapsulated resource prediction service into a component of Gridinformation service, registered to the MDS frame as backend of Gridinformation server, and announced prediction data via a form of resourceinformation service. Thus, we have set up a prototype system of real-timeresource prediction in Computing Grid environment.Finally, we propose the coming future work. Due to various reasons, thereal-time resource prediction system we have implemented in Gridenvironment does not store the measurement data into database, considersonly finite latest measurements when predicting, and only provides predictiondata about resource performance in short time. We should make furtherresearch on prediction about resource availability in long time and provideaccurate prediction data to meet different demands of grid users andapplications. At the same time, we should continue to improve the accuracyof prediction algorithm to provide more effective information of resourceperformance in the future.As a newly emerging technology, the related standards andspecifications of Grid are being established and completed. Real-timeresource prediction service, functioning as a fundamental composition ofGrid research, deserves more applied research and practice.