Complex workflows that process sensor data are useful for industrial infrastructure management and diagnosis. Although running such workflows in clouds promises reduced operational costs, there are still numerous scheduling challenges to overcome. Such complex workflows are dynamic, exhibit periodic patterns, and combine diverse task groupings and requirements. In this work, we propose ANANKE, a scheduling system addressing these challenges. Our approach extends the state-of-the-art in portfolio scheduling for data centers with a reinforcement-learning technique, and proposes various scheduling policies for managing complex workflows. Portfolio scheduling addresses the dynamic aspect of the workload. Q-learning, allows our approach to adapt to the periodic patterns of the workload, and to tune the other configuration parameters. The proposed policies are heuristics that guide the provisioning process, and map workflow tasks to the provisioned cloud resources. Through real-world experiments based on real and synthetic industrial workloads, we analyze and compare our prototype implementation of ANANKE with a system without portfolio scheduling (baseline) and with a system equipped with a standard portfolio scheduler. Overall, our experimental results give evidence that a learning-based portfolio scheduler can perform better and consume fewer resources than state-of-the-art alternatives, in particular for workloads with uniform arrival patterns.
|Title of host publication||14th IEEE Int'l Conference on Autonomic Computing (ICAC)|
|Number of pages||6|
|Publication status||Published - 2017|
|Event||ICAC 2017: 14th International Conference on Autonomic Computing - Columbus, United States|
Duration: 17 Jul 2017 → 21 Jul 2017
|Conference||ICAC 2017: 14th International Conference on Autonomic Computing|
|Period||17/07/17 → 21/07/17|