A Declarative Optimization Engine for Resource Provisioning of Scientific Workflows in IaaS Clouds

Abstract

Resource provisioning for scientific workflows in Infrastructure-as-a-service (IaaS) clouds is an important and complicated problem for budget and performance optimizations of workflows. Scientists are facing the complexities resulting from severe cloud performance dynamics and various user requirements on performance and cost. To address those complexity issues, we propose a declarative optimization engine named Deco for resource provisioning of scientific workflows in IaaS clouds. Deco allows users to specify their workflow optimization goals and constraints of specific problems with an extended declarative language. We propose a novel probabilistic optimization approach for evaluating the declarative optimization goals and constraints in dynamic clouds. To accelerate the solution finding, Deco leverages the available power of GPUs to find the solution in a fast and timely manner. We evaluate Deco with several common provisioning problems. We integrate Deco into a popular workflow management system (Pegasus) and show that Deco can achieve more effective performance/cost optimizations than the state-of-the-art approaches.