Monitoring, configuration and resource management of service workflows in virtualized clusters and clouds

The term Cloud Computing has been used to define commoditized clusters of server nodes where elasticity and on-demand resource provisioning facilitate the efficient processing of user applications and services. A promise of the Cloud Computing paradigm is that workflows can be hosted and managed more efficiently based on the new level of dynamism and control. However, new requirements and challenges arise when leveraging cloud techniques in this manner. This dissertation explores the monitoring, configuration and dynamic resource management of service workflows in a cloud environment. A major contribution is to utilize the dynamism of virtualized cloud resources in various workflow management operations. Several algorithms are proposed throughout the dissertation, each focusing on a different aspect of the larger problem, from monitoring individual services, to placing a new service workflow in the cloud, to dynamically reallocating resources across different services to satisfy demands and reduce costs. The goal is to add an end-to-end solution to the cloud provider's offerings to workflow owners so that the latter can host their workflows in the cloud smoothly without worrying about managing the underlying cloud resources themselves. We show through experimental results, from both real world cluster trace logs and synthetic data, that the proposed approaches can perform various management tasks for service workflows efficiently.