Towards a Resource Manager for Scheduling Frameworks

Aleksandra Kuzmanovska, Rudolf Mak, Dick Epema

Research output: Chapter in Book/Conference proceedings/Edited volumeConference contributionScientificpeer-review

1 Citation (Scopus)

Abstract

Due to the diversity in the applications that run in large distributed environments, many different application frameworks have been developed, such as MapReduce for data-intensive batch jobs and Spark for interactive data analytics. After initial deployment, a framework starts executing a large set of jobs that are submitted over time. When multiple such frameworks with time-varying resource demands are consolidated in a large distributed environment, static allocation of resources on a per-framework basis leads to low system utilization and to resource fragmentation. The goal of my PhD research is to improve the system utilization and framework performances in such consolidated environments by using dynamic resource allocation for efficient resource sharing among frameworks. My contribution towards this goal is a design and an implementation of a scalable resource manager that dynamically balances resources across set of multiple diverse frameworks in a large distributed environment based on resource requirements, system utilization or performance levels in the deployed frameworks.
Original languageEnglish
Title of host publication16th IEEE/ACM Int'l Symp. on Cluster, Cloud and Grid Computing: Doctoral Symposium
Pages592-595
Number of pages4
DOIs
Publication statusPublished - 2016
Externally publishedYes

Fingerprint

Dive into the research topics of 'Towards a Resource Manager for Scheduling Frameworks'. Together they form a unique fingerprint.

Cite this